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Preface 


The  fiscal  year  (FY)  2007  National  Defense  Authorization  Act  (NDAA)  requires  that  cost 
estimates  for  major  defense  acquisition  programs  (MDAPs)  and  major  automated  information 
system  (MATS)  programs  be  performed  by  properly  qualified  members  of  the  armed  forces 
or  full-time  employees  of  the  Department  of  Defense  (DoD).  As  one  step  toward  complying 
with  this  requirement,  the  Air  Force  Assistant  Secretary  for  Acquisition  (SAF/AQ)  directed 
the  Deputy  Assistant  Secretary  for  Acquisition  Integration  (SAF/AQX)  and  the  Air  Force  Cost 
Analysis  Agency  (AFCAA)  to  identify  the  existing  cost-estimating  workforce,  describe  its 
characteristics,  and  assess  its  operational  needs.  In  turn,  SAF/AQX  and  AFCAA  asked  RAND 
Project  AIR  FORCE  to  collaborate  with  them  in  conducting  a  study  of  the  cost-analysis  work¬ 
force  in  the  Air  Force  Materiel  Command’s  (AFMC’s)  product  and  logistic  centers  to  address 
these  questions.  The  study  was  conducted  within  the  Manpower,  Personnel,  and  Training 
Program  of  RAND  Project  AIR  FORCE.  This  technical  report  contains  the  study’s  findings. 
Other  related  RAND  Corporation  cost  studies  include  the  following: 

•  Joseph  G.  Bolten,  Robert  S.  Eeonard,  Mark  V.  Arena,  Obaid  Younossi,  and  Jerry  M. 
Sollinger,  Sources  of  Weapon  System  Cost  Growth:  Analysis  of  35  Major  Defense  Acquisition 
Programs,  Santa  Monica,  Calif:  RAND  Corporation,  MG-670-AF,  2008. 

•  Obaid  Younossi,  Mark  A.  Eorell,  Kevin  Brancato,  Cynthia  R.  Cook,  Mel  Eisman,  Ber¬ 
nard  Fox,  John  C.  Graser,  Yool  Kim,  Robert  S.  Eeonard,  Shari  Eawrence  Pfleeger,  and 
Jerry  M.  Sollinger,  Improving  the  Cost  Estimation  of  Space  Systems:  Past  Lessons  and  Future 
Recommendations,  Santa  Monica,  Calif:  RAND  Corporation,  MG-690-AF,  2008. 

•  Mark  V.  Arena,  Robert  S.  Eeonard,  Sheila  E.  Murray,  and  Obaid  Younossi,  Historical  Cost 
Growth  of  Completed  Weapon  System  Programs,  Santa  Monica,  Calif:  RAND  Corpora¬ 
tion,  TR-343-AF,  2006. 

•  Obaid  Younossi,  Mark  V.  Arena,  Robert  S.  Eeonard,  Charles  Robert  Hall,  Jr.,  Arvind 
Jain,  and  Jerry  M.  Sollinger,  Is  Weapon  System  Cost  Growth  Increasing?  A  Quantitative 
Assessment  of  Completed  and  Ongoing  Programs,  Santa  Monica,  Calif:  RAND  Corpora¬ 
tion,  MG-588-AF,  2006. 


RAND  Project  AIR  FORCE 

RAND  Project  AIR  FORCE  (PAF),  a  division  of  the  RAND  Corporation,  is  the  U.S.  Air 
Force’s  federally  funded  research  and  development  center  for  studies  and  analyses.  PAF  pro- 
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vides  the  Air  Force  with  independent  analyses  of  policy  alternatives  affecting  the  development, 
employment,  comhat  readiness,  and  support  of  current  and  future  aerospace  forces.  Research 
is  conducted  in  four  programs:  Force  Modernization  and  Employment;  Manpower,  Personnel, 
and  Training;  Resource  Management;  and  Strategy  and  Doctrine. 

Additional  information  about  PAF  is  available  on  our  Web  site: 
http://www.rand.org/paf/ 
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IX 


Summary 


Previous  work  by  PAF  has  documented  both  the  creeping  growth  in  the  cost  of  military  acqui¬ 
sition  systems  and  the  factors  that  contribute  to  this  growth,  including  inappropriate  initial 
estimated  costs,  underestimated  technical  risks,  government  decisions  that  led  to  program 
changes,  and  lack  of  independence  of  cost  analysts  from  the  program  offices.  Recommen¬ 
dations  to  improve  cost  estimation  have  included  instituting  independent  program  reviews, 
emphasizing  technical  risk  assessment,  and  requiring  major  estimates  to  be  led  by  experienced 
and  qualified  government  analysts  (Arena  et  ah,  2006;  Younossi  et  ak,  2006  and  2008;  Bolten, 
2008). 

Echoing  some  of  these  recommendations,  in  2007  Congress  dictated  that  cost  estimates 
for  MDAPs  and  MAIS  programs  be  performed  by  properly  qualified  members  of  the  armed 
forces  or  full-time  employees  of  the  DoD  (U.S.  Congress,  2007).  To  provide  information 
toward  eventually  meeting  this  mandate,  PAF  was  asked  to  collaborate  with  SAF/AQX  and 
AFCAA  to  conduct  a  census  of  the  acquisition  cost-estimating  workforce,  describe  its  back¬ 
ground  and  competencies,  and  assess  its  operational  needs. 

In  January  and  February  2008,  we  conducted  a  comprehensive  census  of  the  military, 
government  civilian,  and  contractor  staff  who  self-reported  that  they  were  performing  cost- 
estimation  tasks  in  the  Air  Force’s  four  product  centers  and  three  logistics  centers:  Air  Arma¬ 
ment  Center  (AAC),  Aeronautical  Systems  Center  (ASC),  Electronic  Systems  Center  (ESC), 
Space  and  Missile  Systems  Center  (SMC),  Oklahoma  City  Air  Fogistics  Center  (OC-AFC), 
Ogden  Air  Fogistics  Center  (OO-AFC),  and  Warner-Robins  Air  Fogistics  Center  (WR-AFC). 
We  also  conducted  interviews  with  commanders  in  three  of  the  four  product  centers.*  In  most 
instances,  these  commanders  were  accompanied  by  their  chief  cost  estimator  and/or  chief 
financial  manager.  We  also  held  focus  groups  with  a  sample  of  cost  estimators. 

In  the  interviews  and  focus  groups,  we  asked  about  the  adequacy  of  the  size  and  quality  of 
the  cost-estimating  workforce,  its  desirable  composition  and  mix,  the  background  and  training 
requirements  for  cost  estimators,  and  future  requirements  for  the  cost-estimating  workforce. 


Census  Results 

As  of  winter  2008,  there  were  374  cost  estimators  in  the  seven  Air  Force  product  and  logis¬ 
tics  centers,  split  about  evenly  between  organic  (government  civilian  and  military)  staff  and 
contractors.  In  addition,  there  were  74  unfilled  organic  cost-estimator  positions,  comprising 


*  Per  agreement  with  SAF/AQX,  we  did  not  interview  staff  at  the  smallest  of  the  four  product  centers  and  at  the  logistics 
centers  because  of  the  small  number  of  cost  estimators  in  these  centers. 
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29  percent  of  all  organic  positions.  Nearly  three-quarters  of  these  unfilled  positions  were  for 
civilians,  and  one-quarter  were  for  military  officers  (see  pp.  5-6). 

About  two-thirds  of  cost  estimators  reported  spending  75  percent  or  more  of  their  time 
on  cost-estimating  tasks  over  the  six  months  preceding  the  census.  About  one-quarter  of  cost 
estimators  reported  spending  less  than  50  percent  of  their  time  on  cost-estimating  tasks  (see 
p.  12). 

A  majority  (two-thirds)  of  the  cost-estimating  workforce  had  an  advanced  degree,  typi¬ 
cally  a  master’s  degree  in  business  administration,  finance,  or  accounting.  Fifteen  percent  of 
cost  estimators — nearly  all  of  them  contractors — reported  having  a  degree  in  engineering  or 
mathematics  (see  pp.  8-9). 

One-half  of  the  organic  workforce  had  five  or  fewer  years  of  cost-estimating  experience 
and  30  percent  had  11  or  more  years  of  experience.  In  contrast,  one-half  of  the  contractors  had 
more  than  1 1  years  of  experience,  and  about  one-quarter  had  less  than  five  years  of  experience 
(see  pp.  9-10). 

Nearly  all  (80  percent)  of  the  workforce  reported  having  received  some  formal  cost¬ 
estimating  training,  although  they  reported  that  this  training  was  not  extensive.  The  organic 
workforce  received  most  of  its  training  under  the  Acquisition  Professional  Development  Pro¬ 
gram  (APDP),  which  offered  only  two  short  courses  in  cost  estimation.  The  centers  themselves 
have  no  current  capability  to  offer  additional  training.  Contractors  received  training  from  mul¬ 
tiple  sources,  including  APDP.  Two-thirds  of  the  cost-estimating  workforce,  both  organic  and 
contracted,  reported  having  no  certification  in  cost-estimating  (see  pp.  10-11). 

Forty-five  percent  of  cost  estimators  reported  being  lead  estimators.  There  were  no  marked 
differences  between  organic  lead  estimators  and  organic  estimators  who  were  not  lead  estima¬ 
tors.  About  50  percent  of  both  organic  lead  estimators  and  those  who  were  not  had  five  or 
fewer  years  of  cost-estimating  experience.  In  contrast,  contractor  lead  estimators  were  more 
likely  to  have  1 1  or  more  years  of  cost-estimating  experience  than  those  contractors  who  were 
not  lead  estimators  (68  versus  40  percent)  (see  pp.  13-14). 

Nearly  all  Acquisition  Category  I  (ACAT  I)  programs  had  a  lead  estimator  assigned,  as 
did  two-thirds  of  ACAT  II  programs  and  about  40  percent  of  ACAT  III  programs  (see  pp. 
15-16). 

There  were  notable  variations  across  product  centers  in  the  patterns  described  for  the  cost¬ 
estimating  workforce  as  a  whole,  including  the  following: 

•  The  cost-estimating  workforce  was  primarily  contracted  at  ESC  and  primarily  organic  at 
AAC  and  ASC. 

•  ASC  had  the  largest  proportion  of  cost  estimators  with  a  master  of  business  administra¬ 
tion  (MBA)  degree  (44  percent). 

•  Sixteen  percent  or  less  of  the  cost-estimating  workforce  at  AAC  and  SMC  had  6  to  10 
years  of  experience,  suggesting  that  these  centers  may  encounter  greater  difficulties  than 
other  centers  in  filling  positions  that  require  senior-level  experience. 

•  ESC’s  organic  and  contracted  staff  were  more  likely  than  those  of  other  centers  to  spend 
75  percent  or  more  of  their  time  on  cost-estimating  tasks. 

•  Among  product  centers,  AAC  and  SMC  had  the  lowest  proportion  of  lead  cost  estima¬ 
tors  in  their  workforce  (42  and  27  percent,  respectively),  and  ESC  had  the  highest  (65 
percent). 
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•  None  of  SMC’s  contracted  cost  estimators  were  lead  estimators,  while  at  ESC,  72  per¬ 
cent  of  the  lead  estimators  were  contractors.  AAC  and  ASC  were  also  more  likely  to  use 
organic  staff  than  contractors  as  lead  estimators  (82  and  61  percent,  respectively). 

•  At  SMC,  where  many  lead  estimators  were  military,  nearly  three-quarters  had  five  or 
fewer  years  of  experience. 

•  Only  one-third  of  ACAT  II  programs  were  assigned  a  lead  estimator  at  AAC  and  SMC. 
Almost  two-thirds  of  ACAT  III  programs  at  ESC  had  a  lead  estimator. 


Workforce  Adequacy,  Composition,  and  Competencies 

All  three  centers  where  interviews  were  conducted  relied  primarily  on  contractors  to  meet  their 
cost-estimating  requirements,  regardless  of  the  share  of  organic  cost  estimators  at  the  center. 
ESC,  whose  cost-estimating  workforce  is  less  than  25  percent  organic,  has  little  choice  hut  to 
rely  almost  entirely  on  its  contractors  to  meet  its  cost-estimating  needs.  Although  nearly  two- 
thirds  of  ASC’s  cost-estimating  workforce  is  organic,  its  groups  nevertheless  reported  relying 
primarily  on  contractors  to  provide  leadership  and  experience  to  meet  their  programs’  cost¬ 
estimating  needs.  ASC’s  organic  cost-estimating  staff  are  financial  management  (FM)  general¬ 
ists,  reportedly  to  maintain  flexibility  to  respond  to  day-to-day  work  requirements  and  to  facili¬ 
tate  the  integration  of  cost-estimating  and  FM  functions.  Similarly,  the  actual  cost  estimation 
at  SMC  is  typically  done  by  contractors,  while  organic  staff  are  self-described  “cost  managers” 
who  oversee  several  contractor  cost  estimators  (see  pp.  18-19). 

The  three  centers  estimated  an  aggregate  need  for  a  minimum  of  70  cost-estimator  autho¬ 
rizations,  in  addition  to  the  74  vacant  authorizations  they  currently  hold.  Although  wings  and 
groups  may  hire  additional  contractors  to  fill  some  of  these  gaps,  in  practice  they  said  they  are 
constrained  by  contract  cost  ceilings  set  by  the  center,^  higher  center  or  wing  priorities  on  con¬ 
tractors  in  other  functional  areas  such  as  engineering,  and  the  need  to  use  program  funds  to 
cover  the  costs  of  contractors  (see  pp.  19-20). 

Most  commanders  said  they  would  prefer  to  have  a  higher  ratio  of  experienced  organic 
cost  estimators  to  contractors,  so  that  the  organic  staff  would  have  the  experience  to  take  the 
lead  role  in  performing  cost  estimates  and/or  reviewing  contractors’  estimates.  They  typically 
would  prefer  a  50/50  ratio  of  organic  to  contractor  staff  (see  p.  20). 

Two  workforce-mix  issues  were  raised  by  our  respondents.  The  first  concerns  the  distribu¬ 
tion  of  the  organic  workforce  by  years  of  experience.  One-half  of  organic  cost  estimators  have 
five  or  fewer  years  of  experience  and  one-third  have  more  than  ten  years,  leaving  a  gap  in  the 
middle.  This  gap  is  caused  in  part  by  centers’  inability  to  hire  mid-level  cost  analysts  to  fill  cur¬ 
rently  vacant  positions  and  in  part  by  the  low  retention  rate  of  organic  cost  estimators,  which 
is  attributed  to  a  lack  of  career  opportunities  in  this  specialty.  The  cost-estimating  workforce 
forms  only  a  small  proportion  of  the  FM  field,  which  offers  greater  advancement  opportunities 
in  other  specialties.  Thus,  it  is  unattractive  for  junior  staff  to  enter  a  cost-estimating  career  (see 
pp.  20-21). 

The  second  workforce-mix  issue  concerns  the  trend  toward  the  use  of  organic  FM  gener¬ 
alists  for  cost-estimating  tasks  and  the  lack  of  career  advancement  opportunities  for  cost  esti- 


^  At  ASC,  the  contract  is  centerwide.  At  ESC  and  SMC,  there  are  separate  contracts  for  each  wing,  but  these  centers  have 
authority  to  set  the  contracts’  dollar  ceilings. 
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mators.  Several  commanders  and  chief  cost  estimators  felt  that  cost-estimating  is  a  specialty 
that  requires  more  preparation,  development,  and  experience  than  is  feasible  for  FM  generalists 
(see  p.  21). 

Nevertheless,  respondents  were  generally  satisfied  with  the  competencies  of  their  current 
organic  cost-estimating  workforce,  and  even  more  so  with  their  contracted  cost-estimating 
workforce  (see  p.  21). 

Most  respondents  placed  a  greater  weight  on  years  of  experience  in  practicing  cost¬ 
estimating  and  detailed  knowledge  of  the  programs  than  on  the  background  of  their  cost 
estimators.  Still,  there  was  a  consensus  among  respondents  that  an  engineering  or  technical 
background  is  desirable,  although  it  was  not  deemed  absolutely  necessary.  They  also  said  that 
greater  expertise  was  needed  in  risk  assessment  analysis,  systems  integration,  scheduling,  and 
cost-estimating  for  software  programs  (see  p.  21-22). 

Respondents  said  that  organic  cost  estimators  lacked  adequate  training.  Currently,  most 
of  their  training  occurs  on  the  job  as  they  work  with  experienced  cost  estimators  who  are  pri¬ 
marily  contractors.  Respondents  generally  estimated  that  it  takes  about  five  years  for  a  cost 
estimator  to  become  “competent”  at  performing  cost  estimates  for  complex  programs.  The 
cost-estimating  training  courses  offered  by  the  Defense  Acquisition  University  were  said  to  be 
too  general  and  lack  depth.  Separate  training  courses  are  needed  in  source  selection,  integrated 
baseline  reviews,  program  office  estimates,  risk  analysis,  scheduling,  software  cost-estimating, 
and  operations  and  support.  Respondents  suggested  making  greater  use  of  case  studies  in 
training  courses.  In  contrast,  respondents  praised  the  solid  analytical  preparation  of  Air  Force 
Institute  of  Technology  (AFIT)  graduates,  who  needed  less  time  to  acquire  the  knowledge 
required  for  cost-estimating  (see  pp.  23-24). 

Two  other  issues  require  particular  attention  in  coming  years:  (1)  the  continuing  trend 
toward  more  dependence  on  software  than  on  hardware,  which  will  increase  the  demand  for 
cost  estimators  who  specialize  in  software  programs,  and  (2)  the  desirability  of  reengaging  in 
more-systematic  collection  of  cost  data  and  research  in  cost  estimation.  The  Air  Force  will  need 
to  address  most,  if  not  all,  of  these  issues  in  a  concerted  manner  if  it  is  to  eventually  meet  the 
requirement  of  Section  820  of  the  NDAA  for  cost  estimators. 
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Introduction 


Background 

The  reliability  of  cost  estimates  for  new  weapon  systems  has  long  been  the  subject  of  debate. 
Previous  work  by  RAND  Project  AIR  FORCE  (PAF)  has  concluded  that  the  Department  of 
Defense  (DoD)  and  the  military  departments  have  historically  underestimated  the  cost  of  new 
weapon  systems.  It  has  also  showed  a  trend  of  cost  development  growth  over  the  past  three 
decades  (Arena  et  ah,  2006;  Younossi  et  ah,  2006).  More  recently,  two  RAND  studies,  one  on 
35  completed  major  acquisition  programs  and  the  other  on  Space  and  Missile  Systems  Center’s 
(SMC’s)  cost-estimating  procedures  and  requirements,  identified  several  factors  that  led  to  cost 
growth,  including  inappropriate  initial  estimated  costs,  underestimated  technical  risks,  gov¬ 
ernment  decisions  to  increase  requirements,  increases  in  the  quantity  of  systems  built,  changes 
in  the  program  schedule,  and  lack  of  independence  of  cost  analysts  from  the  program  offices 
(Younossi  et  ah,  2008;  Bolten  et  ah,  2008).  Recommendations  offered  to  improve  cost  estima¬ 
tion  included  instituting  independent  program  reviews,  placing  special  emphasis  on  technical 
risk  assessment,  standardizing  cost  data  collection,  and  requiring  major  estimates  to  be  led  by 
experienced  and  qualified  government  analysts  (Younossi  et  ah,  2008). 

In  2007,  Congress  had  reached  a  similar  conclusion:  that  major  estimates  need  to  be 
led  by  independent,  experienced,  and  qualified  government  analysts.  Section  820  of  the  John 
Warner  National  Defense  Authorization  Act  (NDAA)  for  fiscal  year  (FY)  2007  set  the  follow¬ 
ing  goal: 

It  shall  be  the  goal  of  the  Department  of  Defense  and  the  military  departments  to  ensure 
that  within  five  years  after  the  date  of  the  enactment  of  this  Act,  for  each  Major  Defense 
Acquisition  Program  (MDAP)  and  each  Major  Automated  Information  System  (MAIS), 
each  of  the  following  positions  is  performed  by  a  properly  qualified  member  of  the  Armed 
Forces  or  full-time  employee  of  the  Department  of  Defense:  (f)  Program  Manager;  (2) 
Deputy  Program  Manager;  (3)  Chief  Engineer;  (4)  Systems  Engineer;  (5)  Cost  Estimator. 

In  preparation  to  comply  with  this  requirement,  the  Air  Force  Assistant  Secretary  for 
Acquisition  (SAF/AQ)  directed  the  Deputy  Assistant  Secretary  for  Acquisition  Integration 
(SAF/AQX)  and  the  Air  Force  Cost  Analysis  Agency  (AFCAA)  to  “identify  the  existing  cost¬ 
estimating  work  force  and  capture  the  competencies  of  cost  estimators  supporting  AF  acqui¬ 
sition  programs”  (Durante,  undated).  SAF/AQX  and  AFCAA,  in  turn,  asked  RAND  PAF 
to  collaborate  in  conducting  studies  addressing  these  two  concerns.  Our  mandate  was  (1)  to 
conduct  a  census  of  military  personnel,  government  civilians,  and  contractors  performing  cost 
estimations  and  (2)  to  describe  these  cost  estimators’  qualifications  and  experience  and  the 
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challenges  they  face.  The  Air  Force  intended  to  use  the  findings  from  this  information  gather¬ 
ing  in  support  of  its  decisionmaking  to  comply  with  the  requirement  of  Section  820  of  the  FY 
2007  NDAA  and  to  improve  the  quality  and  reliability  of  cost  estimates. 


Approach 

The  study  was  conducted  in  two  phases.  In  the  first  phase,  a  comprehensive  census  of  all  per¬ 
sonnel  engaged  in  cost-estimating  activities  (military,  civilian,  and  contractors)  was  conducted 
in  January  and  February  2008  in  the  seven  Air  Force  acquisition  product  and  logistics  cen¬ 
ters:  Air  Armament  Center  (AAC),  Aeronautical  Systems  Center  (ASC),  Electronic  Systems 
Center  (ESC),  Space  and  Missile  Systems  Center  (SMC),  Oklahoma  City  Air  Eogistics  Center 
(OC-AEC),  Ogden  Air  Eogistics  Center  (OO-AEC),  and  Warner-Robins  Air  Eogistics  Center 
(WR-AEC).  We  asked  that  the  following  personnel  be  included  in  this  census: 

Any  military  or  government  civilian  engaged  in  acquisition  cost  estimating,  cost  analysis 
and  modeling,  and/or  cost/risk  assessment,  either  full-time  or  part-time;  or  any  contractor 
personnel  or  consultant  working  in  similar  positions  for  an  AF  organization  in  support  of 
acquisition  programs;  or  any  authorized  but  vacant  position  supporting  similar  cost  analy¬ 
sis  and  estimating  (e.g.,  AFSC  [Air  Force  Specialty  Code]  65W  military  positions  in  acqui¬ 
sition  offices).  Individuals  providing  data  in  support  of  cost  analysis  should  only  be  counted 
in  this  category  if  they  participate  in  the  analysis  of  that  data  to  produce  cost  estimates. 
Earned  value  management*  activity  should  only  be  counted  in  this  category  if  it  directly 
supports  cost  estimating  activities.  Also  include  cost  analysts  assigned  to  the  office  who  are 
temporarily  assigned  elsewhere  for  training,  short-term  details  or  other  TDY. 

A  survey  questionnaire  was  developed  to  acquire  the  following  type  of  information  from 
cost  estimators: 

•  Air  Force  organization  and  office  symbol 

•  acquisition  category  (ACAT)^  or  other  programs  they  were  currently  working  on 

•  whether  they  were  working  full-time,  part-time,  or  deployed 

•  contracting  organization  (for  contractor  employees  only) 

•  whether  they  were  a  lead  estimator,  and  if  so,  for  which  program 

•  duty  AFSC 

•  percentage  of  work  time  spent  on  acquisition  cost-estimating,  earned  value  management 
(EVM)  other  than  in  support  of  cost  analysis,  and  other  FM  tasks  during  the  past  six 
months 

•  total  number  of  years  of  cost-estimating  and  other  FM  experience 

•  total  number  of  years  of  acquisition  experience 


*  Earned  value  management  is  a  project  management  technique  that  measures  project  progress  in  an  objective  manner.  It 
combines  the  measurement  of  project  scope,  schedule,  and  cost  in  a  single  integrated  manner.  EVM  is  designed  to  provide 
an  early  warning  of  performance  problems. 

^  ACAT  is  an  acronym  for  a  defense  program  acquisition  category.  The  DoD  divides  acquisition  programs  into  four  acqui¬ 
sition  categories:  ACAT  I,  ACAT  II,  ACAT  III,  and  ACAT  lA.  The  differences  among  these  categories  depend  on  size  and 
programmatic  differences. 
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•  education  and  field  of  study 

•  formal  cost-estimating  training 

•  certification  type  and  source. 

We  also  collected  the  relevant  subset  of  this  information  for  vacant  military  or  civilian 
positions.  The  survey  (see  Appendix  A)  was  administered  at  each  center  by  a  point  of  contact  in 
the  center’s  FM  office.  Survey  responses  were  entered  into  a  spreadsheet  to  facilitate  analysis. 

In  the  second  phase  of  the  study,  we  conducted  supplemental  face-to-face  interviews  with 
selected  commanders  (or  deputy  commanders  if  the  commander  was  not  available)  of  up  to 
seven  wings  or  groups  at  each  of  three  product  centers:  ASC,  ESC,  and  SMC.  The  wings  and 
groups  interviewed  are  listed  in  Appendix  B.  In  addition,  we  conducted  two  focus  groups  at 
each  of  these  centers,  one  with  up  to  four  lead  cost  estimators  and  one  with  up  to  four  other 
cost  estimators.  Interviews  were  conducted  over  two  days  at  each  center  during  the  months  of 
May,  June,  and  July  2008.  For  consistency  across  centers  and  groups,  two  RAND  interviewers 
conducted  all  the  interviews  using  semi-structured  protocols  that  covered  the  following  set  of 
topics  (see  Appendix  C): 

•  characteristics  and  functions  of  the  center’s  Financial  Management  Cost  (FMC) 
division 

•  needs  for  additional  cost  estimators,  by  type 

•  implications  of  not  having  a  sufficient  number  of  cost  estimators  of  the  needed  composi¬ 
tion,  mix,  and  experience 

•  adequacy  of  the  competencies  of  current  cost  estimators  and  views  of  future  require¬ 
ments 

•  adequacy  of  and  need  for  training. 


Organization  of  the  Report 

Chapter  Two  describes  the  characteristics  of  the  current  cost-estimating  workforce  based  on 
the  census  conducted  in  January  and  February  of  2008.  Chapter  Three  discusses  the  current 
and  future  needs  for  additional  staff  and  for  specific  competencies  and  training  identified 
during  our  on-site  interviews.  The  last  chapter  presents  our  overall  conclusions. 


CHAPTER  TWO 


The  Current  Cost- Estimating  Workforce 


This  chapter  presents  the  results  of  a  census  of  staff  who  reported  that  they  worked  on  cost¬ 
estimating  tasks  in  the  Air  Force’s  four  product  and  three  logistics  centers.  The  census  was 
held  during  January  and  February  of  2008.  First,  we  discuss  the  composition  of  the  cost¬ 
estimating  workforce.  Next,  we  describe  the  qualifications  of  the  cost-estimating  workforce 
and  the  amount  of  time  cost  estimators  spent  performing  cost  estimations  around  the  time  of 
the  census.  Finally,  we  discuss  the  characteristics  of  lead  estimators  and  the  ACAT  program  to 
which  they  were  assigned. 

The  reader  should  be  cognizant  of  the  limitations  of  this  census.  First,  it  counted  the  cost¬ 
estimating  workforce  at  a  single  point  in  time  in  the  Air  Force  Materiel  Command’s  (AFMC’s) 
four  product  and  three  logistics  centers.*  In  these  centers,  the  number  of  personnel  engaged 
in  cost  estimating  can  fluctuate  due  to  rotations,  attrition,  hiring,  and  workload  variations 
over  the  programs’  acquisition  cycles.^  Second,  the  census  counted  everyone — military,  civil¬ 
ian,  and  contractor — who  spent  any  time  on  cost-estimating  tasks,  regardless  of  the  amount 
of  time  spent  on  these  tasks  and  regardless  of  the  AFSC  or  other  characteristics  of  the  staff 
included.  Finally,  each  staff  member  self-reported  whether  he  or  she  met  our  definition  of  a  cost 
estimator  and  determined  whether  he  or  she  was  working  on  cost-estimating  tasks. 


Composition  of  the  Cost-Estimating  Workforce 

As  of  February  2008,  there  were  374  military,  government  civilian,  and  contractor  personnel 
working  on  cost-estimating  tasks,  hereafter  referred  to  as  “cost  estimators.”  ASC,  ESC,  and 
SMC  each  had  about  100  cost  estimators,  AAC  had  26  cost  estimators  and  the  three  logistics 
centers  had  15  or  fewer  cost  estimators  each  (Table  2.1). 

Overall,  about  half  of  the  cost-estimating  labor  force  was  organic  (military  and  govern¬ 
ment  civilian)  and  the  other  half  was  contracted  (Figure  2.1).  There  were,  however,  sharp  varia¬ 
tions  across  centers.  At  ESC,  three-quarters  of  the  cost-estimating  labor  force  was  contracted, 
whereas  at  AAC  and  ASC,  73  and  66  percent  of  the  cost-estimating  labor  force  was  organic, 
respectively.  At  SMC,  the  cost-estimating  workforce  was  about  evenly  divided  between  organic 
and  contracted. 


*  In  addition  to  AFMC’s  product  and  logistics  centers,  cost  estimators  are  also  assigned  to  other  Air  Force  organizations, 
including  AFMC  Headquarters  and  AFCAA.  Cost  estimators  in  these  other  organizations  are  not  included  in  this  study. 

^  For  a  description  of  program  acquisition  cycles,  see  Department  of  Defense,  2008. 
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Table  2.1 

Number  and  Percentage  of  Cost  Estimators  and  Vacant  Cost-Estimating  Positions, 
by  Center,  2008 


Filled  Positions 

Vacant  Positions 

Total 

Center 

Number 

Percent 

Number 

Percent 

Number 

Percent 

AAC 

26 

7 

3 

4 

29 

7 

ASC 

117 

31 

21 

28 

138 

31 

ESC 

104 

25 

10 

15 

114 

26 

SMC 

91 

24 

26 

35 

117 

26 

OC-ALC 

12 

3 

5 

7 

17 

4 

00-ALC 

14 

4 

0 

0 

14 

3 

WR-ALC 

10 

3 

9 

12 

19 

4 

Total 

374 

100 

74 

100 

448 

100 

SOURCE:  RAND  census,  2008. 

NOTE:  Percentages  may  not  add  to  100  because  of  rounding. 


Figure  2.1 

Composition  of  the  Cost-Estimating  Workforce,  2008 
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SOURCE:  RAND  census,  2008.  See  also  Appendix  D,  Table  D.1. 
NOTE:  Percentages  may  not  add  to  100  because  of  rounding. 
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The  distribution  of  the  organic  force  between  the  FM  office  and  program  wings^  at  each 
center  differed  as  well.  At  AAC  and  SMC,  about  one-third  of  the  organic  cost  estimators  were 
assigned  to  the  FM  office  and  provided  support  to  the  program  wings  as  center  priorities  dic¬ 
tated.  At  ASC,  a  slightly  higher  share  (42  percent)  of  the  organic  cost-estimating  force  was 
assigned  to  the  FM  office.  In  contrast,  ESC  had  only  10  percent  of  its  cost  estimators  assigned 


^  A  wing  is  a  military  organizational  unit  with  a  specified  mission.  It  is  typically  headed  by  a  colonel.  A  wing  may  be 
divided  into  several  subunits  named  groups. 
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to  the  FM  office.  Furthermore,  nearly  half  of  its  organic  staff  was  off-site,  i.e.,  located  at  Max¬ 
well  Air  Force  Base  (AFB)  and  Wright-Patterson  AFB  rather  than  at  Hanscom  AFB,  which  is 
ESC’s  headquarters.  In  all  centers,  the  central  staff  had  an  oversight  function  as  well. 

At  two  of  the  logistics  centers,  OC-ALC  and  OO-ALC,  the  cost-estimating  workforce 
was  predominantly  organic;  at  WR-ALC,  it  was  predominantly  contracted. 

One  out  of  five  organic  cost  estimators  was  military,  with  a  somewhat  larger  military 
presence  at  AAC  and  SMC  than  at  the  other  five  centers  (see  Figure  2.1). 


Vacant  Positions 

In  winter  2008,  the  centers  reported  having  a  large  number  (74)  of  authorized  but  vacant 
positions  for  organic  cost  estimators — approximately  one-third  of  all  their  organic  positions. 
Almost  two-thirds  of  these  vacant  positions  were  at  ASC  and  SMC  (Table  2.1).  At  SMC,  38 
percent  of  the  organic  positions  were  reported  to  be  vacant;  at  WR-ALC,  more  than  80  percent 
of  organic  positions  were  vacant  (Figure  2.2).  As  discussed  in  the  next  chapter,  respondents 
attributed  this  large  number  of  vacancies  primarily  to  difficulty  in  hiring  organic  staff  at  the 
desired  level  of  experience. 

Seventy-seven  percent  of  vacancies  were  in  civilian  positions  and  23  percent  in  military 
positions  (see  Appendix  D,  Table  D.2).  The  majority  (two-thirds)  of  the  vacant  positions  for 
military  cost  estimators  were  at  the  captain  level,  the  others  were  distributed  between  0-2 
(1st  lieutenant)  and  0-4  (major)  with  one  vacancy  for  an  0-5  (lieutenant  colonel).  On  the 

Figure  2.2 

Percentage  of  Filled  and  Vacant  Organic  Positions  for  Cost  Estimators,  2008 
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(n  =  22)  (n  =  98)  (n  =  35)  (n  =  68)  (n  =  16)  (n  =  8)  (n  =  11) 


SOURCE:  RAND  census,  2008. 
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civilian  side,  the  largest  number  of  vacancies  were  at  what  used  to  be  the  GS-12  (20)  and  GS-13 
(12)  levels/  Five  vacant  positions  were  for  GS-11  and  hve  for  GS-14.  Only  a  few  vacant  posi¬ 
tions  (12)  were  at  the  entry  level  (GS-09  or  below). 


Qualifications  of  the  Cost-Estimating  Workforce 

Education 

About  two-thirds  of  cost  estimators  had  either  a  master  of  business  administration  (MBA) 
degree  (21  percent)  or  a  master’s  degree  in  some  other  held  (42  percent). ^  The  remaining  cost 
estimators  had  bachelor’s  degrees;  only  1  percent  had  a  PhD.  There  were  no  marked  differences 
between  the  organic  workforce  and  contractors  in  their  educational  attainment  (Table  2.2). 

Nor  were  there  any  marked  differences  in  the  share  of  cost  estimators  with  master’s 
degrees  across  centers.  The  only  major  exceptions  were  at  AAG  and  SMG,  where  none  of  the 
cost  estimators  had  an  MBA.  In  contrast,  ASG  had  the  largest  share  of  organic  cost  estimators 
with  MBAs  (44  percent)  (see  Appendix  D,  Table  D.3). 

Few  cost  estimators  had  either  an  engineering  background  (8  percent)  or  a  mathematic 
background  (7  percent);  nearly  all  those  who  did  were  contractors  located  at  ESG  and  SMG. 
One-third  of  contractors  at  SMG  and  20  percent  at  ESG  had  engineering  or  mathematics 
backgrounds.  About  half  of  the  remaining  cost  estimators  had  studied  business  or  manage¬ 
ment,  and  one-quarter  had  studied  finance  or  accounting.  About  one  in  every  ten  cost  estima¬ 
tors  reported  having  an  academic  degree  in  cost  analysis  (figure  2.3). 

Grade 

About  two-thirds  of  government  civilian  cost  estimators  were  at  the  GS-12  and  GS-13  levels. 
Only  about  one  in  ten  was  a  GS-14  or  GS-15.  Similarly,  most  military  cost  estimators  were  in 
the  early  years  of  their  career.  Nearly  two-thirds  were  captains  and  another  23  percent  were 


Table  2.2 

Cost  Estimators'  Educational  Attainment,  by  Personnel  Type,  2008  (%) 


Personnel  Type 

Bachelor's 

Master's 

MBA 

PhD 

Total 

Organic  (n  =  180) 

32 

44 

23 

0 

100 

Contractor  (n  =  186) 

39 

40 

20 

1 

100 

All  (n  =  366)3 

36 

42 

21 

1 

100 

SOURCE:  RAND  census,  2008. 

NOTE:  Percentages  may  not  add  to  100  because  of  rounding. 

^  In  this  and  other  tables  and  figures  in  the  remainder  of  this  document,  "All  n"  may  not  add 
to  374  cost  estimators  because  some  respondents  did  not  provide  all  the  information  asked  for 
in  the  census  survey. 


^  The  U.S.  Government  General  Schedule  (GS)  system  used  to  have  a  pay  scale  from  a  minimum  of  1  to  a  maximum  of 
15,  depending  on  qualifications  and  experience.  It  has  been  replaced  by  the  National  Security  Personnel  System  (NSPS), 
which  has  five  pay  schedules  and  up  to  three  pay  bands  within  each  pay  schedule.  The  NSPS  was  still  being  implemented  as 
of  the  time  of  this  study. 

^  The  Air  Force  has  no  specific  educational  background  requirements  for  cost  estimators  other  than  a  bachelor’s  degree. 
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Figure  2.3 

Backgrounds  of  Cost  Estimators,  2008 
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SOURCE:  RAND  census,  2008. 

NOTE:  Percentages  may  not  add  to  100  because  of  rounding. 
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lieutenants.  Fewer  than  10  percent  of  military  cost  estimators  were  lieutenant  colonels  or  colo¬ 
nels  (Figure  2.4).  The  tendency  of  the  organic  cost-estimating  workforce  to  he  concentrated  in 
relatively  low  grades  reflects  the  lack  of  advancement  opportunities  said  to  he  available  in  this 
career  field.  In  our  interviews,  the  lack  of  a  career  path  and  advancement  opportunities  were 
frequently  mentioned  hy  respondents  as  problems  for  recruiting,  hiring,  and  retaining  cost  esti¬ 
mators  (see  Chapter  Three).  However,  three-quarters  of  contractors  rated  themselves  as  senior, 
with  only  about  one  in  four  contracted  cost  estimators  rating  themselves  as  junior  or  mid-level. 


Experience 

There  were  sharp  differences  in  the  amount  of  cost-estimating  experience  of  organic  and  con¬ 
tractor  cost  estimators.  About  50  percent  of  the  organic  workforce  had  five  or  fewer  years  of 
experience;  of  those,  two-thirds  had  two  or  fewer  years  of  experience.  In  contrast,  half  of  the 
contractors  had  11  or  more  years  of  experience,  and  half  of  these  had  21  or  more  years  of  expe¬ 
rience.  About  20  percent  of  both  the  organic  and  contractor  workforce  had  about  6  to  10  years 
of  experience  (Table  2.3). 

Military  personnel  tended  to  have  the  fewest  years  of  cost-estimating  experience.  Two- 
thirds  of  military  cost  estimators  had  less  than  two  years  of  experience  and  only  16  percent  had 
six  or  more  years  of  experience  (Table  2.4). 

Across  centers,  there  were  only  two  notable  exceptions  to  the  pattern  described  above: 
Sixteen  percent  or  less  of  the  organic  cost-estimating  workforce  at  AAC  and  SMC  had  6  to 
10  years  of  experience,  suggesting  that  they  may  encounter  greater  difficulties  than  other  cen¬ 
ters  in  filling  positions  requiring  senior-level  experience  in  the  future  (see  Appendix  D,  Table 
D.4). 
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Figure  2.4 

Grades  and  Types  of  Cost  Estimators,  2008 
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SOURCE:  RAND  census,  2008. 

NOTE:  Percentages  may  not  add  to  100  because  of  rounding. 
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Table  2.3 

Cost  Estimators'  Years  of  Experience,  by  Personnel  Type,  2008  (%) 


Personnel  Type 

0-2 

3-5 

6-10 

11-20 

21  + 

Total 

Organic  (n  =  180) 

33 

18 

19 

18 

11 

100 

Contractor  (n  =  178) 

13 

14 

23 

26 

24 

100 

All  (n  =  358) 

23 

16 

21 

22 

17 

100 

SOURCE:  RAND  census,  2008. 

NOTE:  Percentages  may  not  add  to  100  because  of  rounding. 


Table  2.4 

Military  Cost  Estimators'  Years  of  Experience,  by  Product  Center,  2008  (%) 


Center 

0-2 

(n  =  26) 

3-5 
(n  =  7) 

6-10 
(n  =  3) 

11-20 
(n  =  11) 

"  t 

Total 

AAC  (n  =  6) 

17 

0 

33 

17 

33 

100 

ASC  (n  =  7) 

100 

— 

— 

— 

— 

100 

ESC  (n  =  9) 

67 

22 

11 

0 

0 

100 

SMC  (n  =  17) 

71 

29 

— 

— 

— 

100 

All  (n  =  39) 

67 

18 

8 

3 

5 

100 

SOURCE:  RAND  census,  2008. 


Training 

About  80  percent  of  the  cost-estimating  workforce  reported  having  received  some  type  of 
formal  training  in  cost  estimation.  Nearly  all  civilians  (96  percent)  reported  having  received 
formal  training,  but  only  two  of  every  three  contractors  did  so  (Table  2.5). 
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Table  2.5 

Cost  Estimators'  Training,  by  Personnel  Type,  2008  (%) 


Training 

Military 
(n  =  40) 

Civilian 
(n  =  144) 

Contractor 
(n  =  178) 

Total 
(n  =  362) 

APDP  1 

48 

13 

5 

13 

APDP  II 

13 

21 

5 

12 

APDP  III 

5 

47 

13 

25 

Air  Force  Institute  of 
Technology  (AFIT) 

15 

3 

3 

4 

Automated  Cost  Estimating 
Integrated  Tools  (ACEIT) 

0 

1 

12 

6 

Tecolote^  training 

0 

0 

10 

5 

Other 

3 

13 

24 

18 

No  training 

18 

4 

29 

18 

Total 

100 

100 

100 

100 

SOURCE:  RAND  census,  2008. 

NOTE:  When  respondents  indicated  more  than  one  type  of  training,  we  selected  the  highest  level  of 

training  provided.  Percentages  may  not  add  to  100  because  of  rounding. 

^  Tecolote  is  the  name  of  the  contracted  company  providing  cost  estimators. 

Training  provided  under  the  Acquisition  Professional  Development  Program  (APDP)*^ 
was  the  most  frequent  form  of  formal  training  received  hy  military  (66  percent)  and  civilian 
(81  percent)  cost  estimators.  Whereas  most  military  personnel  had  received  APDP  I  training 
only,  the  majority  of  civilians  had  received  APDP  III  training  because  they  had  been  in  the 
field  long  enough  to  become  trained  and  certified  at  Level  III.  Most  of  the  military  did  not 
have  the  experience  to  go  beyond  Level  IJ 

Contractors,  in  contrast,  reported  receiving  several  types  of  training  if  they  had  received 
any.  About  one  in  four  reported  receiving  APDP  training,  probably  when  they  were  in  the 
military  or  were  government  civilian  employees.®  About  half  had  received  training  from  other 
sources,  including  AFIT,  ACEIT,^  or  the  contracting  agency  that  employed  them.  The  remain¬ 
ing  29  percent  reported  having  received  no  training. 

Certification 

Two-thirds  of  the  cost-estimating  workforce,  including  military,  civilian,  and  contractor  per¬ 
sonnel,  reported  having  no  certification  in  cost-estimating.  The  remainder  reported  various 
types  of  certification,  the  most  frequent  being  Certified  Cost  Estimator/Analyst  (CCE/A, 


^  APDP  is  a  certification  process  involving  required  training  and  years  of  experience  to  be  certified  at  the  different  levels.  It 
offers  Air  Force  personnel  courses  and  certification  in  FM,  contracting,  systems,  engineering,  program  management,  logis¬ 
tics,  information  technology,  and  scientific  research  and  development.  It  has  basic,  mid-level,  and  advanced  courses  (Levels 
1  through  Ill)  in  these  areas. 

^  We  did  not  collect  information  on  what  courses  they  had  taken  or  the  content  of  the  courses.  It  is  probably  fair  to  say 
that  the  training  in  cost  estimating  received  from  the  APDP  programs  was  minimal  because  few  courses  in  cost  estimating 
are  offered  by  APDP. 

®  Contractors  are  not  eligible  for  APDP  training. 

^  Automated  Cost  Estimating  Integrated  Tools  are  software  programs  that  support  cost  and  financial  analysts  and  pro¬ 
gram  managers  (PMs).  Training  for  using  these  tools  may  have  been  provided  by  ACEIT  and  by  the  Air  Eorce  product  and 
logistics  centers. 
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awarded  by  the  Society  of  Cost  Estimating  and  Analysis)  (15  percent)  and  Certified  Defense 
Financial  Manager  (CDFM,  including  Certified  Government  Financial  Manager)  (5  percent). 
Again  there  were  some  variations  among  the  military,  civilian,  and  contractor  cost  estimators 
(Table  2.6). 


Time  Spent  on  Cost-Estimating  Tasks 

Respondents  were  asked  how  much  of  their  work  time  they  spent  doing  cost-estimating  tasks 
over  the  previous  six  months.  This  is,  of  course,  a  single-point-in-time  estimate.  The  amount  of 
time  spent  over  the  years  on  cost-estimating  tasks  by  our  respondents  may  have  varied  greatly, 
depending  on  the  status  of  the  program(s)  they  worked  on.  But,  when  averaged  over  all  pro¬ 
grams  and  cost  estimators,  as  we  did  here,  these  estimates  should  be  good  indicators  of  the 
aggregate  time  the  workforce  as  a  whole  spent  on  cost-estimating  tasks. 

About  one  in  every  three  cost  estimators,  either  organic  or  contractor,  had  worked  full¬ 
time  on  cost-estimating  tasks  over  the  previous  six  months.  Of  the  remainder,  contractors  gen¬ 
erally  reported  spending  more  time  on  cost-estimating  tasks  than  did  organic  cost  estimators. 
Contractors  were  more  than  three  times  more  likely  to  have  spent  75  to  99.9  percent  of  their 
time  on  cost-estimating  tasks  than  were  the  organic  staff  (47  versus  14  percent).  And  they  were 
half  as  likely  as  organic  cost  estimators  to  have  spent  less  than  25  percent  of  their  time  on  cost¬ 
estimating  tasks  (Table  2.7). 


Table  2.6 

Cost  Estimators'  Certification,  by  Personnel  Type,  2008  (%) 


Certification 

Military 
(n  =  40) 

Civilian 
(n  =  144) 

Contractor 
(n  =  190) 

Total 
(n  =  374) 

None/unknown 

75 

69 

67 

69 

AFIT 

5 

3 

3 

3 

CCE/A 

10 

11 

19 

15 

CDFM 

8 

8 

2 

5 

Other 

3 

8 

10 

5 

Total 

100 

100 

100 

100 

SOURCE:  RAND  census,  2008. 

NOTE:  Percentages  may  not  add  to  100  because  of  rounding. 


Table  2.7 

Time  Spent  Doing  Cost  Estimation  Over  Previous  Six  Months,  by  Personnel  Type,  2008  (%) 


Personnel  Type 

None 

Less  Than 

25 

25  to 
49.9 

50  to 
74.9 

75  to 
99.9 

100 

Total 

Organic  (n  =  184) 

5 

17 

17 

12 

14 

35 

100 

Contractor  (n  =  190) 

1 

10 

2 

8 

47 

32 

100 

All  (n  =  374) 

3 

13 

9 

10 

32 

33 

100 

SOURCE:  RAND  census,  2008. 

NOTE:  Percentages  may  not  add  to  100  because  of  rounding. 
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While  the  pattern  at  each  individual  center  is  generally  similar  to  that  of  the  cost¬ 
estimating  force  as  a  whole,  there  are  a  number  of  notable  exceptions.  In  ESC,  both  organic 
and  contractor  staff  were  more  likely  than  in  other  product  centers  to  spend  most  of  their  time 
on  cost-estimating  tasks.  In  that  center,  more  than  75  percent  of  the  organic  staff  and  more 
than  90  percent  of  contractors  spent  75  percent  or  more  of  their  time  on  cost-estimating  tasks. 
And  at  SMC,  organic  cost  estimators  were  the  least  likely  of  the  organic  staff  at  any  center  to 
spend  more  than  75  percent  of  their  time  on  cost-estimating  tasks;  only  about  one-third  did 
so  (see  Appendix  D,  Table  D.5). 

Finally,  at  two  of  the  logistics  centers,  OO-ALC  and  WR-ALC,  most  (50  percent  or 
more)  of  the  organic  cost-estimating  staff  were  spending  less  than  25  percent  of  their  time  on 
cost-estimating  tasks,  as  were  contractors  at  the  WR-ALC  center. 

Lead  Estimators'" 

Overall,  about  half  of  the  cost-estimating  staff  reported  that  they  were  lead  cost  estimators, 
i.e.,  the  person  responsible  for  ensuring  the  completeness  and  reliability  of  the  cost  estimates. 
Most  military  cost  estimators  (75  percent)  reported  that  they  functioned  as  lead  estimators 
while  about  half  (45  percent)  of  government  civilians  and  contractors  reported  being  lead 
estimators. 

The  composition  (military,  government  civilian,  and  contractor)  of  lead  estimators 
and  their  proportion  of  the  cost-estimating  workforce  varied  greatly  across  centers.  Among 
product  centers,  AAC  and  SMC  had  the  lowest  proportions  of  lead  estimators  in  their  cost¬ 
estimating  workforce  (43  and  27  percent,  respectively)  and  ESC  had  the  highest  share  (66 
percent).  Among  logistics  centers,  OC-ALC  had  one  lead  estimator  (9  percent)  while  50  and 
30  percent  of  the  cost-estimating  workforce  at  OO-ALC  and  WR-ALC,  respectively,  reported 
they  were  lead  estimators  (Figure  2.5). 

The  proportion  of  contractors  functioning  as  lead  estimators  varied  across  centers  as  well. 
None  of  the  contracted  cost  estimators  at  SMC  reported  functioning  as  lead  estimators;  this 
center  relied  primarily  on  military  personnel,  and  secondarily  on  civilians,  to  perform  its  lead¬ 
estimating  functions.  In  contrast,  72  percent  of  ESC’s  lead  cost  estimators  were  contractors 
(see  Appendix  D,  Table  D.6).  Both  AAC  and  ASC  relied  primarily  on  their  civilian  labor  force 
to  perform  lead  cost-estimating  functions,  with  ASC  less  likely  than  AAC  to  use  civilian  staff 
(57  versus  64  percent). 

Experience  of  Lead  Estimators 

The  distribution  of  organic  lead  estimators’  years  of  experience  mirrored  that  of  organic  cost 
estimators  who  were  not  lead  estimators.  About  40  percent  of  lead  and  other  cost  estimators 
had  fewer  than  five  years  of  experience.  In  contrast,  contractor  lead  estimators  were  generally 
more  experienced  than  other  contractor  estimators;  about  two-thirds  of  contractor  lead  estima¬ 
tors  had  1 1  or  more  years  of  experience  compared  with  40  percent  of  other  contractor  estima¬ 
tors  (Table  2.8). 


Although  Section  820  of  the  FY  2007  NDAA  specifies  that  cost  estimators’  functions  must  be  performed  by  “a  prop¬ 
erly  qualified  member  of  the  Armed  Forces  or  full-time  employee  of  the  Department  of  Defense”  within  five  years  after 
enactment  of  the  Act,  SAF/AQX  staff  indicated  that  the  intent  of  the  legislation  is  to  apply  mainly  to  lead  cost  estimator 
functions. 
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Figure  2.5 

Cost  Estimators  Who  Are  Lead  Estimators,  by  Center  and  Type  of  Personnel,  2008 


□  Contractor 

□  Government 
civilian 

■  Military 


SOURCE:  RAND  census,  2008. 

RAND  TR70S-2.S 


Table  2.8 

Lead  Estimators'  Years  of  Experience  in  Cost  Estimation,  by  Center  and  Personnel  Type, 
2008  (%) 


Personnel  Type 

0-2 

3-5 

6-10 

11-20 

21  + 

Total 

Organic 

Lead  estimator  (n  =  89) 

34 

22 

18 

15 

10 

100 

Other  estimator  (n  =  91) 

36 

12 

20 

23 

9 

100 

Contractor 

Lead  estimator  (n  =  78) 

4 

12 

22 

35 

28 

100 

Other  estimator  (n  =  100) 

19 

15 

25 

19 

21 

100 

All 

Lead  estimator  (n  =  167) 

20 

17 

20 

24 

19 

100 

Other  estimator  (n  =  191) 

27 

14 

23 

21 

15 

100 

SOURCE:  RAND  census,  2008. 

NOTE:  Percentages  may  not  add  to  100  because  of  rounding. 


At  SMC,  where  many  lead  estimators  are  military,  three-quarters  had  five  or  fewer  years 
of  experience.  In  contrast,  ESC’s  lead  estimators  were  primarily  contractors,  and  over  half  of 
them  had  11  or  more  years  of  experience  (see  Appendix  D,  Table  D.7). 


Time  Worked  on  Cost-Estimating  Tasks 

Overall,  lead  cost  estimators  were  slightly  more  likely  to  work  full-time  on  cost-estimating 
tasks  in  the  previous  six  months  than  were  other  cost  estimators  (37  percent  versus  29  per- 
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cent).  Similarly,  lead  cost  estimators  were  less  likely  to  spend  less  than  25  percent  of  their  time 
on  cost-estimating  tasks  than  were  other  cost  estimators  (10  percent  versus  24  percent)  (Table 
2.9). 


ACAT  Programs  and  Lead  Estimators 

We  examined  the  proportion  of  ACAT  I,  II,  and  III  programs  that  had  an  assigned  lead  esti¬ 
mator,  based  on  a  list  of  ACAT  programs  provided  by  SAF/AQX  as  of  January  2008.  Of  the 
196  ACAT  programs  on  this  list,  53  were  ACAT  I,''  29  were  ACAT  II,  and  114  were  ACAT 
III  (Table  2.10). 

Overall  and  at  the  time  of  our  survey  (winter  2008),  57  percent  (112)  of  the  ACAT  pro¬ 
grams  listed  had  a  lead  estimator. All  ACAT  I  programs  at  AAC  and  SMC  had  a  lead  cost 
estimator  assigned  to  them,  and  nearly  all  such  programs  had  a  lead  estimator  at  ASC  and  ESC 
(87  and  94  percent,  respectively)  (Table  2.11). 

ACAT  II  programs  were  less  likely  than  ACAT  I  programs  to  have  a  lead  cost  estimator 
assigned — just  over  60  percent  had  one.  However,  there  were  large  variations  across  centers: 
At  ASC,  80  percent  of  ACAT  II  programs  had  a  lead  estimator;  only  one-third  of  ACAT  II 
programs  at  AAC  and  SMC  had  a  lead  cost  estimator  assigned. 

About  40  percent  of  ACAT  III  programs  had  a  lead  cost  estimator  assigned — again  with 
large  variations  across  centers.  At  ESC,  nearly  two-thirds  of  ACAT  III  programs  had  a  lead 
cost  estimator,  compared  with  43  and  22  percent  of  ACAT  III  programs  at  SMC  and  ASC, 
respectively.  Fewer  than  one  of  every  six  ACAT  III  programs  at  AAC  had  a  lead  cost  estimator 
assigned. 


Table  2.9 

Lead  Estimators'  Time  Spent  Doing  Cost  Estimation,  by  Personnel  Type,  2008  (%) 


Personnel  Type 

None 

Less  Than 
25 

25 

to  49.9 

50 

to  74.9 

75 

to  99.9 

100 

Total 

Organic 

Lead  estimator  (n  =  90) 

1 

10 

9 

19 

19 

42 

100 

Other  estimator  (n  =  94) 

9 

24 

25 

4 

10 

28 

100 

Contractor 

Lead  estimator  (n  =  79) 

0 

4 

3 

10 

52 

32 

100 

Other  estimator  (n  =  111) 

1 

15 

3 

6 

44 

31 

100 

All 

Lead  estimator  (n  =  169) 

1 

7 

6 

15 

34 

37 

100 

Other  estimator  (n  =  205) 

4 

20 

13 

6 

28 

29 

100 

SOURCE:  RAND  census,  2008. 

NOTE:  Percentages  may  not  add  to  100  because  of  rounding. 


''  These  ACAT  I  programs  include  ACAT  I  and  ACAT  I  A. 

Of  the  169  self-reported  lead  estimators,  126  reported  working  on  an  ACAT  program  (a  program  may  have  more  than 
one  lead  estimator),  three  led  resource  selection,  and  five  were  branch  or  section  chiefs.  The  remaining  number  reported 
being  lead  estimators  on  programs  not  included  on  the  SAF/AQX  list  provided  for  this  study. 
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Table  2.10 

Number  of  ACAT  Programs,  by  Center  and  ACAT  Program  Level 


Center 

ACATI 

ACAT  II 

ACAT  III  Total 

AAC 

5 

3 

12 

20 

ASC 

15 

15 

49 

79 

ESC 

18 

5 

39 

62 

SMC 

15 

6 

14 

35 

All 

53 

29 

114 

196 

SOURCE:  List  of  programs  provided  by  SAF/AQX,  January  2008, 

Table  2.11 

ACAT  Programs  with  a  Lead  Estimator,  by  Center  and  ACAT  Program 
Level,  2008  (%) 

Center 

ACAT  1 
(n  =  53) 

ACAT  II 
(n  =  29) 

ACAT  III 
(n  =  114) 

All 

AAC  (n  =  20) 

100 

33 

17 

40 

ASC  (n  =  79) 

87 

80 

22 

46 

ESC  (n  =  62) 

94 

60 

64 

73 

SMC  (n  =  35) 

100 

33 

43 

66 

All  (n  =  196) 

94 

62 

39 

57 

SOURCE:  RAND  census,  2008. 


CHAPTER  THREE 


Cost- Estimating  Workforce  Adequacy,  Composition,  and 
Competencies 


We  conducted  face-to-face  interviews  in  the  spring  and  summer  of  2008  with  wing  or  group 
commanders  at  three  of  the  seven  centers  included  in  the  study:  seven  group  or  squadron 
commanders  at  ASC,  hve  group  commanders  at  ESC,  and  six  wing  commanders  at  SMCd  In 
most  instances,  these  commanders  were  accompanied  hy  their  chief  cost  estimator  and/or  chief 
financial  manager.  In  addition,  we  interviewed  the  chief  of  the  FMC  division  and  held  two 
focus  groups  in  each  center,  one  with  up  to  four  lead  cost  estimators  and  the  other  with  up  to 
four  other  cost  estimators. 

In  the  interviews  and  focus  groups,  we  asked  for  the  participants’  assessment  of  the  ade¬ 
quacy  of  their  cost  analysis  workforce,  both  quantitatively  and  qualitatively.  If  wing  and  group 
commanders  indicated  that  their  needs  for  cost  estimators  were  not  being  met,  we  asked  how 
they  coped  to  meet  their  programs’  cost-estimating  needs.  Respondents  were  also  asked  about 
their  perspectives  on  their  cost-estimator  workforce  composition  and  mix;  the  desirable  com¬ 
petencies,  hiring  and  training  requirements  for  cost  estimators;  and  future  requirements  for  the 
cost-estimating  workforce  (see  interview  protocol  in  Appendix  C). 

When  interpreting  the  findings  below,  the  reader  should  keep  in  mind  that  they  reflect  the 
opinions  of  our  respondents.  With  few  exceptions,  we  did  not  make  independent  checks.  Also, 
we  focus  in  this  report  primarily  on  characteristics  and  issues  for  which  there  was  relatively 
strong  agreement  across  most  respondents  and  across  all  three  centers.  In  the  few  instances 
where  there  was  a  marked  diversion  of  opinions,  we  indicate  so.  Our  purpose  is  to  provide  an 
aggregate  overview  of  the  reported  strengths  and  weaknesses  of  the  cost-estimating  workforce 
as  a  whole;  it  is  not  to  expose  individual  views,  however  strongly  held. 

This  chapter  first  describes  the  characteristics  of  the  three  product  centers’  cost  divisions, 
which  are  designed  to  support  and  oversee  the  cost-estimating  activities  of  wings  and  groups 
on  behalf  of  their  respective  program  executive  officers  (PEOs).  Next,  we  discuss  the  adequacy 
of  the  cost-estimating  workforce  in  terms  of  its  quantity,  mix,  and  competencies,  and  look  at 
the  coping  mechanisms  used  to  address  problems  in  any  of  these  areas.  We  then  outline  a  set  of 
issues  raised  by  our  respondents  that  should  be  addressed  in  order  to  improve  the  effectiveness 
of  the  cost-estimating  workforce.  These  include  hiring,  training,  data  collection,  and  mainte¬ 
nance.  We  conclude  with  a  brief  discussion  of  future  trends  mentioned  by  our  respondents  that 
will  need  to  be  addressed  in  coming  years. 


*  Per  agreement  with  SAF/AQX  and  AFCAA,  we  did  not  conduct  interviews  at  AAC  and  the  logistics  centers  because  they 
have  fewer  cost  estimators. 
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Roles  and  Capabilities  of  the  Central  Cost-Estimating  Divisions 

All  three  product  centers  maintain  a  central  cost  staff  (the  FMC  division)  that  is  responsible  to 
the  center’s  PEO,  although  the  manning  of  the  divisions  differs  signihcantly.  ASC’s  cost  staff  is 
the  largest,  with  32  organic  cost  estimators;  SMC’s  has  12  cost  estimators,  75  percent  of  whom 
are  organic;  and  ESC’s  is  the  smallest  with  only  ten  cost  estimators  at  the  time  of  our  inter¬ 
views.  ESC’s  division  was  being  reestablished  after  having  been  abolished  when  authority  and 
responsibilities  over  staff  were  transferred  from  the  functional  chiefs  to  the  wing  commanders 
(Air  force  Materiel  Command,  2004). 

The  cost  staff  described  their  main  functions  in  an  overlapping  manner,  including 

•  formulating  and  implementing  center  policy  on  cost-estimating 

•  conducting  peer  reviews  and  serving  as  an  honest  broker 

•  conducting  independent  cost  studies  and  analyses 

•  assisting  wings  during  surge  needs  for  cost  estimates. 

Although  manned  at  widely  different  levels,  all  three  centers  reported  that  they  did  not 
have  a  sufficient  number  of  staff  to  perform  these  and  other  functions.  To  hll  these  gaps,  ESC/ 
EMC  reported  that  it  had  received  authorizations  to  hire  15  additional  organic  staff  over  the 
next  three  years  and  indicated  needing  another  5  to  15  additional  staff.  ASC/f  MC  reported 
that  it  had  obtained  authorizations  for  ten  staff  in  addition  to  the  ten  vacancies  it  currently  had. 
SMC  personnel  said  they  needed  30-35  additional  positions  centerwide. 

Respondents  identihed  three  issues  related  to  their  estimated  shortages  of  cost 
estimators: 

First,  the  FMC  divisions  are  unable  to  provide  sufficient  cost-estimating  surge  assistance  to 
their  respective  wings  and  groups.  Even  ASC/f  MC,  which  reported  that  it  typically  assigns  85  to 
90  percent  of  its  relatively  large  staff  of  cost  estimators  to  surge  activities  at  any  one  time,^  said 
it  was  unable  to  meet  all  requests  from  wings  and  groups.  SMC/EMC  and  ESC/EMC  indi¬ 
cated  that  they  were  not  able  to  provide  surge  assistance,  an  assessment  that  was  also  confirmed 
by  their  centers’  wings  and  groups. 

Second,  none  of  the  centers  had  developed  an  internal  capacity  to  train  newcomers  or  provide 
ongoing  training  to  keep  cost  estimators  abreast  of  new  cost- estimating  practices  and  to  share  “best 
practices.  ” 

Third,  the  centers’  cost  staffi had  little  or  no  capacity  to  do  research  and  collect,  update,  and 
share  cost  data  to  provide  a  better  basis  for future  cost  models.  The  only  exception  was  SMC/EMC, 
which  had  recently  assigned  two  organic  cost  estimators  to  form  a  research  unit  with  the  pri¬ 
mary  function  of  collecting  and  normalizing  cost  data  on  newer  technologies. 


Adequacy  of  the  Cost-Estimating  Workforce 

All  three  centers  rely  primarily  on  contractors  to  meet  their  cost-estimating  requirements, 
regardless  of  the  proportion  of  organic  cost  estimators  at  the  center.  ESC,  whose  cost- 


^  Reportedly,  ASC/FMC  devotes  little  time  to  its  other  functions  including  cost  studies  and  analyses,  updating  cost  pro¬ 
cesses  and  tools,  and  the  like. 
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estimating  workforce  is  less  than  25  percent  organic,  has  little  choice  hut  to  rely  almost  entirely 
on  its  contractors  to  meet  its  cost-estimating  needs.  Even  ASC,  where  nearly  two-thirds  of  the 
cost-estimating  workforce  is  organic,  nevertheless  relies  primarily  on  contractors  to  provide 
leadership  and  experience  to  meet  their  cost-estimating  needs.  The  organic  cost  estimators  in 
ASC’s  groups  are  FM  generalists,  reportedly  to  maintain  flexibility  to  respond  to  day-to-day 
work  requirements  and  to  facilitate  the  integration  between  cost-estimating  and  FM  functions. 
Half  of  this  staff  has  less  than  two  years  of  cost-estimating  experience.  Similarly,  at  SMC  the 
actual  cost-estimating  is  typically  done  by  contractors.  Organic  staff  are  self-described  “cost 
managers”  who  oversee  several  contractor  cost  estimators  to  make  sure  that  cost  estimates  are 
done  on  time;  however,  they  also  perform  other  FM  functions  as  well. 

In  our  interviews,  several  respondents  told  us  that  the  depletion  of  their  organic  cost¬ 
estimating  capability  originated  with  a  decision  made  in  the  1990s  to  contract  out  the  acquisi¬ 
tion  cost-estimating  functions.  Since  then,  a  cost-estimating  career  has  become  unattractive 
for  organic  staff  because  of  the  lack  of  advancement  opportunities  in  the  small  organic  cost¬ 
estimating  workforce  and  the  low  esteem  in  which  this  specialization  is  held,  according  to  our 
respondents.  As  a  result,  more  than  50  percent  of  organic  cost  estimators  have  five  or  fewer 
years  of  experience  and  only  30  percent  have  more  than  ten  years  of  experience.  Organic  staff 
tend  to  leave  cost-estimating  positions  after  a  few  years  if  they  desire  to  improve  their  promo¬ 
tion  opportunities. 


Need  for  Additional  Cost  Estimators 

Nearly  all  our  respondents  said  they  needed  additional  staff  to  fully  meet  all  their  cost¬ 
estimating  needs.  A  few  respondents,  however,  were  reluctant  to  say  they  needed  more  cost 
estimators,  indicating  that  they  were  able  to  “get  things  done”  or  that  “it  is  a  subjective  matter 
whether  or  not  they  had  enough  staff.”  Currently,  there  is  no  objective  yardstick  or  methodol¬ 
ogy  to  assess  the  need  for  cost  estimators  that  accounts  for  an  acquisition  program’s  type,  size, 
and  stage  in  the  acquisition  life  cycle.  ^ 

Together,  the  three  centers  estimated  an  aggregate  need  for  a  minimum  of  70  additional 
organic  staff.  ASC  estimated  that  it  needed  14  to  34  additional  authorizations;  SMC  estimated 
a  need  for  30  to  35  additional  organic  staff.  ESC  did  not  provide  a  centerwide  estimate,  but 
estimates  from  individual  commanders  totaled  about  27  additional  positions.  These  needs  were 
in  addition  to  the  57  vacancies  at  the  three  centers  that  they  were  already  having  difficulties 
filling.^ 

The  additional  staff  was  needed  for  a  variety  of  tasks,  including 

•  performing  pre-Milestone  A  and  B  estimates 

•  ensuring  that  all  estimates  are  updated  annually 


^  One  commander  volunteered  that  it  takes  one  person-month  to  do  a  program  objective  memorandum  (POM)  request, 
three  to  four  person-months  for  a  Milestone  A  program  office  estimate  (POE),  and  seven  to  eight  months  for  a  Milestone  B 
POE  with  a  Cost  Analysis  Requirements  Description  (CARD).  Milestone  A  is  a  review  occurring  after  a  material  solution 
has  been  adopted  and  prior  to  the  technology  development  strategy  phase.  Milestone  B  is  a  review  occurring  at  completion 
of  the  technology  development  strategy  phase  and  prior  to  the  engineering  and  manufacturing  development  phase.  CARD 
describes  the  system’s  mission,  certification,  relationship  to  other  systems,  and  the  major  factors  that  influence  its  costs. 

^  We  discuss  barriers  to  hiring  in  greater  detail  below. 
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•  supporting  all  ACAT  programs 

•  supporting  source  selection 

•  providing  surge  support  to  wings 

•  independently  verifying  and  validating  all  estimates  and  reviewing  their  documentation 

•  verifying  contractor  cost  estimates 

•  overseeing  contractor  cost  systems  and  procedures 

•  collecting,  normalizing,  and  storing  cost  data 

•  providing  recurring,  focused,  in-house  training 

•  enabling  more  specialization. 

Although  wings  and  groups  may  hire  additional  contractors  to  fill  some  of  these  gaps,  in 
practice  they  said  they  are  constrained  hy  center-level  cost  ceilings  on  contracts;  ^  higher  center 
or  wing  priorities  on  contractors  in  other  functional  areas,  such  as  engineering;  and  the  need 
to  use  program  funds  to  cover  the  costs  of  contractors. 


Force  Composition 

Most  commanders  said  they  would  prefer  to  have  a  more  balanced  ratio  of  experienced  organic 
and  contractor  cost  estimators  so  that  the  organic  staff  would  have  the  experience  to  take  the 
lead  role  in  performing  cost  estimates  and/or  reviewing  contractors’  estimates.  They  typically 
preferred  a  50/50  ratio  of  organic  to  contractor  staff,  with  a  few  respondents  preferring  a  75/25 
ratio. 

Some  commanders  also  indicated  a  preference  for  an  all-civilian  organic  cost-estimating 
workforce.  Military  cost  estimators  were  seen  as  less  experienced,  prone  to  be  deployed  (mostly 
as  financial  officers  rather  than  as  cost  estimators),  and  less  likely  to  pursue  a  career  in  cost 
estimation.  There  was  recognition,  however,  that  cost-estimating  experience  may  be  useful  for 
officers  who  later  fill  military  command  positions. 

Several  reasons  were  offered  in  support  of  a  higher  share  of  experienced  organic  cost  esti¬ 
mators.  They  are  needed  to  lead  and  direct  cost  estimates,  exercise  oversight,  provide  continuity,'’ 
and  improve  control  of  requests.  In  addition,  organic  staff  can  be  moved  more  easily,  as  needed 
(especially  to  meet  surges),  and  have  greater  credibility  with  other  organizations,  especially 
with  the  Office  of  the  Secretary  of  Defense  (OSD). 

Force  Mix 

A  Mid-Level  Gap?  Commanders  expressed  concern  that  there  were  too  few  mid-level  cost 
estimators,  not  only  among  the  organic  workforce  but  also  among  contracted  cost  estimators, 
to  replace  those  who  will  soon  retire.  The  organic  cost-estimating  workforce  is  split  between 
the  junior  level  (51  percent)  and  the  senior  level  (29  percent),  and  the  contracted  workforce 
is  primarily  tilted  toward  the  senior  level  (50  percent),  as  discussed  in  Chapter  Two.  SMC’s 
organic  cost-estimating  workforce  is  even  more  unbalanced  with  66  percent  at  the  junior  level 
and  only  10  percent  at  the  mid  level. 


^  At  ASC  and  SMC  contracts  are  centerwide  and  at  ESC  there  are  separate  contracts  for  each  wing.  However,  the  center 
has  authority  to  limit  the  dollar  value  of  the  contracts. 

However,  it  is  not  uncommon  for  contracted  cost  estimators  to  stay  for  several  years  in  a  specific  program  or  wing. 
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The  seniority  distribution  of  the  organic  cost-estimating  workforce  raises  two  issues.  First, 
to  the  extent  that  there  is  a  real  mid-level  gap  in  the  distribution  of  cost  estimators,  it  could 
be  reduced  by  ensuring  the  retention  and  greater  specialization  of  the  current  disproportionate 
reservoir  of  junior-level  cost  estimators.^  Second,  the  small  reservoir  of  mid-level  staff  will  make 
it  very  difficult,  if  not  impossible,  to  fill  vacancies  (and  to  meet  needs  for  additional  staff).  ®  As 
noted  in  Chapter  Two,  the  mid-level  organic  cost-estimating  positions  are  disproportionately 
vacant.  Because  there  are  too  few  potential  candidates  at  this  level,  at  least  under  the  current 
pay  scale,  one  center’s  wing  or  contractor  loss  is  another  center’s  wing  or  contractor  gain.® 

Generalists  Versus  Specialists.  Respondents  held  different  preferences  as  to  whether  cost 
estimators  should  be  generalists,  i.e.,  those  who  can  perform  cost-estimating  as  well  as  other 
FM  tasks,  or  specialists,  i.e.,  those  who  have  more  years  of  experience  and  in-depth  knowledge 
of  cost  estimation.  Some  commanders  and  FM  chiefs  valued  the  flexibility  of  generalists,  who 
can  be  allocated  to  a  variety  of  FM  functions  according  to  day-to-day  program  needs.  Other 
commanders  and  chief  cost  estimators  felt  that  cost  estimation  requires  more  preparation, 
development,  and  experience  than  is  feasible  for  generalists,  especially  as  the  complexity  of  the 
systems  being  developed  increases.  They  also  value  cost  estimators  who  can  “ask  the  right  in- 
depth  questions”  and  perform  independent  estimates,  as  well  as  provide  effective  oversight  over 
contractors.  In  practice,  organic  FM  generalists  need  to  be  coupled  with  expert  cost  estimators, 
who  are  now  most  frequently  contractors. 

There  was  consensus  that  the  FM  field  had  become  more  generalized  over  the  years  and 
seemingly  favored  FM  and  budget  over  cost  estimation. 

Workforce  Competencies 

Respondents  were  generally  satisfied  with  the  competencies  of  their  organic  cost-estimating 
workforce — even  more  so  with  their  contracted  cost-estimating  workforce.  Most  placed  greater 
weight  on  years  of  experience  practicing  cost  estimation  and  on  having  detailed  knowledge  of 
their  programs  than  they  did  on  the  academic  background  of  their  cost  estimators.  Still,  there 
were  areas  where  respondents  thought  more  expertise  was  needed. 

There  was  a  consensus  among  our  respondents  that  an  engineering  or  technical  back¬ 
ground,  although  not  absolutely  necessary,  would  be  particularly  useful  for  understanding 
the  technologies  involved  in  acquisition  programs,  staying  abreast  of  new  developments,  and 
obtaining  more-detailed  information  from  engineers.  Estimators  with  these  backgrounds  were 
thought  to  be  able  to  ask  deeper  questions  than  those  with  an  accounting  or  financial  back¬ 
ground.  A  mathematics  or  operations  research  background  was  also  seen  as  good  preparation 
for  a  cost-estimating  career. 

In  addition,  respondents  mentioned  several  areas  where  more  expertise  was  desirable. 
These  included  risk  assessment  analysis,  systems  integration,  and  analytical  and  conceptual 
skills.  Cost  estimators  also  need  to  know  more  about  acquisition  scheduling  because  of  the  key 
role  it  plays  in  making  good  cost  estimates.  At  SMC,  a  prior  understanding  of  the  space  envi- 


^  Determining  whether  there  are  enough  mid-level  staff  to  eventually  replace  retiring  senior-level  staff  would  require  pro¬ 
jecting  retention  rates  and  retirement  rates  over  the  next  10-20  years. 

®  We  did  not  ask  about  the  seniority  of  the  additional  staff  needed,  but  our  sense  is  that  centers  typically  would  prefer  to 
hire  candidates  with  prior  experience  in  cost-estimating. 

®  We  discuss  this  issue  in  greater  detail  below. 
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ronment  was  deemed  desirable.  In  all  centers,  prior  experience  in  acquisition,  whether  or  not 
in  cost  estimation,  was  also  deemed  important. 

Finally,  there  was  a  near  consensus  that  the  cost-estimating  workforce  as  a  whole  needs  to 
have  more  capability,  both  quantitatively  and  qualitatively,  in  the  use  of  cost-estimating  soft¬ 
ware  programs. 

Coping  with  an  Inadequate  Cost-Estimating  Workforce 

Respondents  used  numerous  methods  to  cope  with  an  undermanned  cost-estimating  work¬ 
force,  including  the  following: 

•  Prioritizing.  Respondents  frequently  reprioritize  to  meet  the  most  pressing  demands  for 
cost  estimates.  In  this  process,  estimates  that  go  to  the  PEO  get  the  highest  priority.  The 
rest  are  delayed  or  do  not  get  done.  ACAT  III  programs  get  low  priority. 

•  Tasks  left  undone.  These  include  yearly  reviews,  integrated  baseline  reviews  (IBRs)  on 
major  programs,  verification  of  contractors’  estimates,  cost  drills,  tracking  of  ongoing 
contracts,  collection  of  background  data,  research  and  data  analysis,  and  staff  develop¬ 
ment.  Also,  operations  and  support  (O&S)  issues  are  often  left  unaddressed. 

•  Use  of  undesirable  practices.  These  include  decisions  and  budgeting  based  on  back-of- 
the-envelope  estimates;  risks  and  uncertainties  not  pursued  systematically;  estimates  left 
unchecked,  relying  on  the  Defense  Contract  Audit  Agency  (DCAA)  to  catch  potential 
problems;  small  programs  relying  on  the  cost  estimates  of  the  product  suppliers;  spe¬ 
cialization  in  cost  analysis  being  discouraged;  and  inexperienced  staff  serving  as  lead 
estimators. 

•  Acceptance  of  increased  risks.  As  a  result,  programs  may  fail  to  recognize  cost  overruns 
on  time,  and  program  and  requirement  inefficiencies  may  increase. 

•  Lower-quality  estimates.  Program  cost  estimates,  mid-term  IBRs,  and  cost-risk  assess¬ 
ments  are  less  accurate  and  defensible. 

•  Staff  burnout.  Rapid  operational  tempo  leads  to  burnout  of  some  staff. 


Barriers  to  Hiring  Cost  Estimators 

As  suggested  by  the  large  number  of  vacancies  in  cost-estimating  positions  at  the  three  cen¬ 
ters  (see  Chapter  Two),  respondents  indicated  that  they  had  difficulties  hiring  cost  estimators. 
These  difficulties  were  attributed  to  several  factors. 

At  ESC  and  SMC,  the  foremost  factor  was  the  high  cost  of  living  in  the  Boston  and  Eos 
Angeles  areas,  respectively.  This  problem  is  greatest  for  entry-level  staff,  but  even  senior  staff 
were  said  to  be  reluctant  to  move  to  these  areas. 

Competition  with  contractors  for  cost  estimators  was  a  pronounced  issue  at  all  three 
centers.  Many  of  the  civilian  vacancies  are  at  the  GS-12  level  or  above,  and  the  centers  most 
often  obtain  contracted  cost  estimators  at  the  mid  and  senior  level.  Hence,  both  the  centers 
and  their  contractors  are  competing  for  candidates  from  the  same  small  pool  of  experienced 
cost  estimators. 


Note  that  we  were  not  able  to  ascertain  the  frequency  with  which  these  coping  mechanisms  were  used. 
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Indeed,  the  Air  Force  Personnel  Center  (AFPC)  was  said  to  have  difficulties  identifying 
government  or  outside  candidates  with  the  desired  cost-estimating  experience.  This  problem 
was  also  attributed,  in  part,  to  job  descriptions  that  are  too  general,  resulting  in  candidates 
with  few  or  none  of  the  specific  attributes  desired.*'  AFPC  was  also  said  to  take  seven  or  more 
months  to  complete  the  hiring  process  of  outside  candidates,  so  some  candidates  may  take 
other  jobs  because  they  cannot  wait  that  long. 

Lack  of  a  career  path  for  cost  estimators  was  also  frequently  mentioned  as  a  barrier  to 
attracting  and  retaining  civilian  cost  estimators.  Because  of  the  small  size  of  the  organic  cost¬ 
estimating  labor  force,  greater  career  advancement  opportunities  are  available  in  other  func¬ 
tions,  such  as  budget  and  FM;  this  creates  an  incentive  for  staff  to  avoid  specializing  in  cost¬ 
estimating.  As  staff  move  up  the  FM  career  ladder,  they  were  said  to  have  a  tendency  to  leave 
cost-estimating  in  favor  of  other  FM  functions  that  offer  many  more  positions  throughout 
government. 

The  one  exception  to  this  pattern  is  entry-level  hiring  at  ASC.  ASC  indicated  that  it  had 
no  difficulty  hiring  recent  college  graduates  at  the  junior  level.  The  center  has  established  close 
relationships  with  colleges  in  the  region  from  which  it  gets  most  of  its  entry-level  hires.  But, 
at  the  mid  and  senior  levels,  ASC  reported  having  difficulties  similar  to  those  at  the  other 
centers.  The  only  option  it  saw  was  to  hire  from  other  centers  or  programs — a  procedure  that, 
if  successful,  only  transfers  the  problem  from  one  location  to  another  within  the  acquisition 
community. 


Training 

The  centers  currently  provide  neither  formal  training  nor  mentoring  to  develop  the  skills  of 
cost  estimators.  Reportedly,  they  do  not  have  the  resources  to  support  an  ongoing  internal 
training  program. 

In  practice,  training  of  organic  staff  occurs  mostly  on  the  job,  as  they  work  with  mid- 
and  senior-level  cost  estimators.  The  “trainers”  were  said  to  be  most  frequently  senior-level 
contracted  cost  estimators.  But  because  of  the  high  operations  tempo,  respondents  also  said 
that  senior  cost  estimators  had  little  time  to  mentor  younger  colleagues  or  permit  trainees  to 
shadow  more- experienced  cost  estimators.  Respondents  typically  estimated  that  it  takes  up  to 
five  years  to  train  a  junior  staff  member  to  become  competent  to  perform  complex  independent 
cost  estimates.  Some  said  that  it  takes  even  longer. 

The  consensus  of  our  respondents  was  that  organic  cost  estimators  are  not  adequately 
trained.  Although  all  staff  in  the  FM  specialty  are  encouraged  to  participate  in  the  Air  Force 
Financial  Management  Training  Program,  not  all  do  so  because  it  is  not  mandated.  And  even 
if  they  do  participate,  courses  in  cost-estimating  are  currently  offered  only  at  the  basic  or  inter¬ 
mediate  level  as  required  courses.  Although  the  Defense  Acquisition  University  (DAU)  offers 


"  Our  reviewer  indicated  that  part  of  the  problem  is  that  AFPC  identifies  candidates  who  do  not  have  the  experience 
needed  because  their  job  experiences  have  been  coded  incorrectly,  e.g.,  coded  as  having  30  percent  cost,  30  percent  EVM, 
and  30  percent  budget  experience,  when  actually  their  experience  was  100  percent  budget. 

Some  respondents  indicated  that  the  operations  tempo  in  their  center  is  so  high  that  they  typically  would  not  have  time 
to  attend  training  sessions  even  if  they  were  offered.  Also,  SMC  sponsors  basewide  training  on  cost-estimating  software 
programs  such  as  SEER  by  Galorath,  PRICE,  ACE  Risk  Management,  and  ACEIT. 
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many  other  elective  courses  in  its  Core-Plus  program  (i.e.,  BCF  206,  Cost/Risk  Analysis;  BCF 
207,  Economic  Analysis;  BCF  208,  Software  Cost  Estimating;  BCE  215,  Operating  and  Sup¬ 
port  Cost  Analysis;  and  BCE  263,  Principles  of  Schedule  Management),  they  usually  are  not 
taken  because  they  are  not  required  for  Business,  Cost  Estimating,  and  Einancial  Manage¬ 
ment  (BCEEM)  certification.  Therefore,  there  is  little  evidence  to  assure  the  employer  that  the 
individual  has  been  trained  on  the  requisite  cost  knowledge.  Consequently,  over  the  past  few 
years  DAU  and  senior  leaders  across  DoD  have  taken  action  to  improve  training  across  the  cost 
estimating  acquisition  workforce  to  assess  the  needs  of  the  BCEEM  community.  This  resulted 
in  the  restructure  of  the  BCEEM  acquisition  community. 

This  restructure  (effective  October  1,  2009)  established  two  separate  career  paths  (Busi¬ 
ness-Cost  Estimating  and  Business-Einancial  Management)  and  renamed  the  BCEEM  career 
field  Business.  This  will  ultimately  satisfy  the  DoD  cost  community’s  internal  needs  for  more 
robust  training  in  the  areas  of  statistics,  cost  risk  analysis,  O&S  cost  estimating,  software 
cost  estimating,  data  collection,  and  analysis  while  complying  with  Section  820  of  the  2007 
NDAA,  Government  Performance  of  Critical  Acquisition  Positions. 

In  contrast,  respondents  generally  praised  the  solid  analytical  preparation  of  graduates 
of  AEIT,  which  shortened  the  time  it  took  them  to  acquire  the  knowledge  needed  to  perform 
cost  estimates.  However,  the  master’s  program  offered  by  AEIT  is  primarily  limited  to  military 
personnel,  and  few  are  given  the  opportunity  to  attend.  Also,  AE IT-trained  military  personnel 
rarely  spend  more  than  one  tour  of  duty  in  cost  analysis  positions. 

Most  respondents  thought  that  better  training  and  mentoring  is  needed  in  several  areas, 
including  source  selection,  IBR,  POEs,  risk  analysis,  scheduling,  software  cost  estimation,  and 
O&S.  They  recommended  that  separate  courses  be  developed  in  these  subjects.  Respondents 
also  suggested  making  greater  use  of  case  studies  in  training  courses. 

finally,  some  respondents  suggested  reestablishing  an  intern  program  and  providing  cost 
estimators  with  opportunities  for  education  with  industry  (EWI)  assignments,  during  which 
they  could  obtain  a  better  understanding  of  budgeting  and  cost-estimating  practices  in  private 
industry. 


Data  Collection  and  Maintenance 

The  validity  of  cost  estimates  depends  in  large  measure  on  the  availability  and  quality  of  data 
collected  from  previous  acquisition  programs  and  cost  estimates.  Our  respondents  said  that 
data  collection  takes  a  significant  portion  of  the  time  involved  in  making  a  cost  estimate,  par¬ 
ticularly  for  estimates  done  early  in  the  life  cycle  of  a  program.  Yet  respondents  indicated  that 
cost-estimate  documentation  was  not  being  emphasized,  shared,  and  made  available  as  back¬ 
ground  data  for  future  cost  estimates.  Also,  there  was  no  exchange  of  data  and  information 
across  programs.  Even  when  cost  information  is  documented,  it  is  not  put  in  a  format  that  is 
easily  accessible  and  usable.  Another  important  issue  that  affects  data  collection  is  the  elimina¬ 
tion  of  data  requirements  in  acquisition  contracts:  Contract  cost  data  have  thereby  lost  consis¬ 
tency  across  programs  and  are  either  unavailable  or  must  be  purchased  from  the  contractors  at 
additional  (and  unplanned)  expense. 


Communication  with  the  Defense  Acquisition  University. 
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Respondents  blamed  lack  of  resources  in  FMC  divisions  and  programs  for  failure  to  per¬ 
form  this  important  task  well,  if  at  all.  As  noted  earlier,  SMC  has  set  up  a  small  research  unit 
in  its  FMC  division  to  collect  cost  data,  focusing  particularly  on  new  technologies. 

Opinions  diverged  on  who  should  be  responsible  for  data  collection  and  maintenance. 
Some  respondents  thought  this  function  should  be  done  centrally  at  the  Air  Staff  level  while 
others  thought  it  should  be  done  at  the  center  level  because  the  types  of  programs  and  tech¬ 
nologies  used  differ  across  centers. 


Looking  Ahead 

Respondents  identified  several  issues  that  will  need  to  be  addressed  in  the  coming  years.  Fore¬ 
most  among  them  is  the  expected  continuing  trend  toward  programs  that  are  more  dependent 
on  software  than  on  hardware.  This  shift,  they  said,  will  require  the  development  of  new  cost¬ 
estimating  techniques  and  more-specialized  cost  estimators.  Respondents  thought  that  many 
cost  estimators  did  not  have  the  required  expertise  to  work  on  these  software  programs. 

Another  issue  identified  is  the  need  to  develop  a  capability  to  estimate  operation  and 
maintenance  (O&M)  costs  for  space  systems.  Reportedly,  O&M  costs  for  operating,  trouble¬ 
shooting,  and  responding  to  anomalies  in  launched  space  systems  have  increased  rapidly — 
from  20  percent  to  40-50  percent  of  system  costs.  Currently,  estimates  of  O&M  costs  come 
entirely  from  contractors,  without  independent  government  reviews. 

Other  forward-looking  issues  identified  by  our  respondents  included  the  following: 

•  A  need  for  niche  expertise  in  the  cost-estimating  workforce.  Respondents  identified 
risk  analysis  and  scheduling  as  areas  that  need  specialized  expertise  that  can  be  shared 
across  programs. 

•  Guidance  and  staff  for  sufficiency  reviews.'^  Current  guidance  was  viewed  as  being 
inconsistent.  Some  programs  (those  that  do  not  have  much  actual  cost  data)  do  not  really 
need  a  sufficiency  review.  Respondents  noted  that  they  did  not  have  enough  staff  with  the 
required  experience  to  perform  sufficiency  reviews. 

•  Staff  development.  Certification  for  cost  analysts  should  be  more  rigorous  to  improve 
the  quality  of  estimators,  and  a  career  path  for  cost  analysis  is  needed  to  attract  and  retain 
cost  analysts.  There  is  also  a  need  for  more  cost  estimators  with  foreign  military  sales 
(FMS)  experience  to  adequately  cover  the  growing  numbers  of  NATO  and  other  pro¬ 
grams  with  FMS. 

•  Use  of  red  teams.  ESC  has  used  red  teams  (which  include  senior  cost  estimators  not 
affiliated  with  the  program)  in  at  least  four  cases  in  the  past  and  has  found  them  useful 
as  well  as  insightful.  Greater  use  of  red  teams,  perhaps  with  industry  representatives,  was 
seen  as  one  way  of  improving  the  quality  and  reliability  of  cost  estimates. 

•  Data  collection.  Data  requirements  should  be  included  in  acquisition  contracts  *5  to 
ensure  that  cost  data  are  available  on  current  programs  and  for  future  estimates. 


A  sufficiency  review  is  a  review  of  program  office  cost  estimates  for  reasonableness,  completeness,  consistency,  and  compli¬ 
ance  with  generally  accepted  estimating  processes.  It  provides  decisionmakers  with  an  assessment  of  the  quality  of  the  cost 
estimate  (Independent  Program  Assessment  Office,  undated). 

SMC  reported  that  it  has  started  to  include  data  requirements  in  contracts  for  new  programs. 
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•  Cost-estimating  research.  Research  has  payoffs  in  improving  cost  estimates,  especially 
given  the  shift  toward  programs  that  are  more  reliant  on  software.  Increasing  research  is 
likely  to  require  a  combination  of  more  organic  staff  and  more  funding  devoted  to  this 
purpose.  Respondents  felt  that  most  research  should  he  conducted  and  supported  at  the 
center  level  rather  than  at  the  wing  level  or  at  AFCAA. 


Closing  Thoughts 

Cost  estimators  who  participated  in  our  focus  groups  emphasized  that  the  quality  of  their  cost 
estimates  depended  in  part  on  the  information  and  data  they  received  from  other  team  mem¬ 
bers.  Some  felt  they  did  not  get  enough  detailed  information  from  engineers  to  make  good 
estimates,  especially  in  software- development  costs,  where  they  have  less  experience. 

At  one  center,  a  major  complaint  was  directed  at  PMs.  Respondents  said  that  PMs  did  not 
have  adequate  knowledge  of  the  acquisition  process  and  were  poor  planners.  Respondents  at 
two  centers  mentioned  frequent  problems  with  scheduling  (incomplete  schedules  or  frequently 
changed  schedules).  As  a  result,  cost  estimators  were  often  given  too  little  time  to  perform 
quality  cost  estimates.  Some  respondents  felt  that  large  programs  should  have  a  designated 
scheduler. 


CHAPTER  FOUR 

Conclusions 


This  report  provides  a  count  of  the  military,  government  civilian,  and  contracted  staff  engaged 
in  cost-estimating  tasks  during  January  and  February  2008  at  the  seven  Air  Force  acquisition 
product  and  logistic  centers.  It  also  describes  the  results  of  interviews  with  wing  and  group 
commanders,  selected  staff,  and  cost  estimators  at  three  product  centers,  in  which  respondents 
assessed  the  adequacy  of  the  acquisition  cost-estimating  workforce,  both  quantitatively  and 
qualitatively.  We  draw  four  overarching  conclusions  from  our  descriptive  findings. 

First,  the  Air  Force  acquisition  product  and  logistics  centers  rely  primarily  on  the  expe¬ 
rience  and  knowledge  of  contractors  to  perform  cost  estimates.  The  organic  staff  is  generally 
too  inexperienced,  or  is  otherwise  needed  to  perform  other  FM  tasks,  to  take  a  leading  role  in 
developing  cost  estimates.  Although  lead  estimators  for  ACAT  I  programs  may  be  nominally 
organic,  the  actual  cost  estimating  may  be  performed  mainly  by  contractors. 

Second,  the  organic  cost-estimating  workforce  seems  to  be  understaffed.  Nearly  all  our 
respondents  said  they  needed  additional  staff  to  fully  meet  all  their  cost-estimating  needs. 
Meeting  these  needs  is  hampered  for  two  reasons:  (1)  The  centers  do  not  have  enough  authori¬ 
zations  for  cost  estimators,  and  (2)  they  are  unable  to  fill  the  vacant  authorizations  they  already 
have.  Centers  encounter  multiple  difficulties  when  hiring  organic  cost  estimators.  At  current 
salaries,  the  high  cost  of  living  is  a  major  barrier  for  both  junior  and  senior  staff  at  ESC  and 
SMC.  Also,  there  are  simply  not  enough  qualified  cost  estimators  (many  of  the  vacant  posi¬ 
tions  are  for  mid-level  or  higher  positions).  As  a  result,  the  centers  compete  among  themselves 
and  with  contractors  to  hire  qualified  cost  estimators.  Finally,  the  Air  Force  is  not  developing 
enough  experienced  organic  cost  estimators.  Because  the  cost  estimation  specialty  is  small  and 
embedded  within  the  FM  field,  which  offers  greater  advancement  opportunities  in  other  spe¬ 
cialties,  it  is  unattractive  for  junior  staff  to  enter  and  stay  in  a  cost-estimating  career.  In  addi¬ 
tion,  the  Air  Force  seems  ambivalent  between  developing  FM  generalists  with  some  knowledge 
of  cost-estimating  or  cost-estimating  specialists  with  in-depth  knowledge  and  multiple  years 
of  experience. 

Third,  organic  cost  estimators  are  not  adequately  trained.  The  centers’  FMC  divisions  do 
not  have  the  resources  to  formally  train  their  staff,  and  the  few  cost-estimating  courses  that  are 
available  at  DAU  were  said  to  be  inadequate  and  insufficient.  Training  of  organic  cost  estima¬ 
tors  is  done  on  the  job,  mainly  by  the  contractors  with  whom  they  typically  work.  Respondents 
generally  estimated  that  it  takes  about  five  years  for  a  cost  estimator  to  become  “competent” 
to  perform  cost  estimates  for  complex  programs.  Organic  cost  estimators  need  more  training 
in  source  selection,  IBR,  POE,  risk  analysis,  scheduling,  software  cost-estimating,  and  O&S. 
Existing  training  courses  could  be  improved  by  making  greater  use  of  case  studies. 
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Finally,  our  respondents  identified  three  other  issues  that  appear  to  need  particular  atten¬ 
tion  in  the  coming  years.  The  first  is  the  expected  continuing  trend  toward  programs  that 
are  more  dependent  on  software  than  on  hardware.  This  shift,  they  reported,  will  require  the 
development  of  new  cost-estimating  techniques  and  more-specialized  cost  estimators.  Second, 
the  Air  Force  needs  to  develop  the  capability  to  estimate  O&M  costs  for  space  systems.  O&M 
costs  for  operating,  troubleshooting,  and  responding  to  anomalies  in  launched  space  systems 
have  increased  rapidly  in  recent  years  and  O&M  cost  estimates  currently  come  entirely  from 
contractors,  without  independent  government  reviews.  Third,  the  centers  should  reengage  in 
cost-estimating  research  and  more-systematic  collection  of  cost  data.  Respondents  felt  that 
these  activities  would  have  payoffs  in  improving  cost  estimates,  especially  given  the  trend 
toward  more  software-dependent  programs. 

The  Air  Force  will  need  to  address  most,  if  not  all,  of  these  issues  in  a  concerted  manner, 
if  it  is  to  meet  the  requirements  of  Section  820  of  the  2007  NDAA  for  cost  estimators. 
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Census  of  Cost  Estimators:  Questionnaire 


Data  Elements  Information  Requested 


1  AF  Organization/  Office  name 

2  ACAT  Program(s)  worked  on  during  past 
6  months 

3  Name  of  Employee 

4  Position  Number  (MPCN) 

5  Status 

6  Lead  Estimator?  For  what  programs? 

7  Mil  Grade  or  Civ  Pay  Plan-Series-Grade 

8  Duty  AFSC 

9  Duty  title 

10  %  time  spent  in  acquisition  cost 
estimating  during  past  six  months 

11  %  time  spent  in  doing  EVM  other  than  in 
support  of  cost  analysis  during  the  past 
six  months 

12  %  time  spent  doing  other  financial 
management  during  past  6  months 

13  Total  Years  of  Cost  Analysis  Experience 

14  Total  Years  of  Other  FM  Experience 


Wing,  group,  squadron,  center  or  other 
organization  identifier,  plus  office  symbol  where 
the  cost  analyst  works. 

Use  acronym(s)  from  Program  Acronyms  sheet 
(worksheet  4)  or  name  of  other  program(s)  if  not 
listed  there. 

Full  name  of  employee. 

Manpower  Position  Control  Number  for  the 
employee. 

Full-time,  on-duty  (default);  Full-time,  TDY 
elsewhere;  part-time. 

List  program  acronyms  or  names  (from  worksheet 
4)  for  which  this  individual  is  the  lead  cost 
estimator.  Leave  blank  or  enter  "No"  if  not  a  lead 
cost  estimator. 

Current  grade  (and  Occupational  Series)  for  the 
employee. 


Official  duty  title  (will  be  extracted  from  personnel 
database  if  left  blank). 

Enter  one  of  the  following  five  categories:  100%; 
75%  or  more;  50%  or  more;  25%  or  more;  less  than 
25%.  Based  on  the  last  6  months  work  or  since 
starting  current  position  if  less  than  6  months. 

Enter  one  of  the  following  four  categories:  75%  or 
more;  50%  or  more;  25%  or  more;  less  than  25%. 
Based  on  the  last  6  months  work  or  since  starting 
current  position  if  less  than  6  months. 

Enter  one  of  the  following  four  categories:  75%  or 
more;  50%  or  more;  25%  or  more;  less  than  25%. 
Based  on  the  last  6  months  work  or  since  starting 
current  position  if  less  than  6  months. 

Count  all  years  the  individual  was  employed  in  cost 
analysis  work  at  least  50%  of  the  time. 

Count  all  years  the  individual  was  employed  in 
FM  work  at  least  50%  of  the  time  including  cost 
estimating  and  analysis. 


29 


30  The  Acquisition  Cost-Estimating  Workforce:  Census  and  Characteristics 


15 

Total  Years  of  Acquisition  Experience 

Count  all  years  the  individual  worked  in  acquisition 
at  least  50%  of  the  time,  including  both  acquisition 
cost  analysis  and  other  acquisition-related 
assignments. 

16 

Highest  Degree  Obtained 

17 

Field  of  Study  for  Highest  Degree 

18 

Formal  Cost  Analysis  Training 

For  example:  APDP  Level  III;  CostProf  (enter 
"NONE"  if  none). 

19 

Certification(s)  and  Source 

For  example:  SCEA,  ISPA  (enter  "NONE"  if  none). 

20 

Notes 

Other  information  or  clarification,  if  needed. 

Instructions  for  filling  out  questionnaire: 

Purpose 

SAF/AQX,  in  cooperation  with  the  RAND  Corporation,  is  conducting  a  study  of  the  status 
of  the  cost  analysis  workforce  Air  Force-wide.  The  purposes  of  the  study  are  to  develop  a  reli¬ 
able  count  of  the  current  cost  analysts  working  on  AF  ACAT  I,  II,  III  programs  and  assess 
the  operating  needs  for  cost  analysts  currently  and  in  the  near  future.  The  list  of  your  Center’s 
programs  is  attached. 

To  achieve  this  goal,  we  need  you  to  hll  in  the  attached  spreadsheet  requesting  specific 
information  on  each  individual  cost  analyst  working  on  one  or  more  of  your  Center’s  ACAT 
programs.  The  information  we  collect  will  he  kept  conhdential  and  none  of  the  information 
provided  on  any  individual  will  he  released  to  anyone  or  he  accessible  to  anyone  else  other  than 
the  RAND  staff  working  on  the  study.  The  information  collected  will  be  used  to  describe  the 
characteristics  and  experience  of  the  current  cost  analyst  workforce  in  the  aggregate. 

Three  separate  worksheet-forms  need  to  be  filled  out  for  (1)  currently  assigned  (govern¬ 
ment  employee)  cost  analysts  (see  definition  below),  (2)  contractor/consultant  cost  analysts, 
and  (3)  vacant  cost  analyst  positions.  Enter  the  information  requested  for  each  individual  cost 
analyst  (and  vacant  position)  in  successive  columns  from  left  to  right;  you  may  make  dupli¬ 
cate  copies  of  the  worksheet-forms  and  use  separate  forms  for  different  offices  if  that  is  more 
convenient. 

Definition  of  Costs  Analysts  for  the  Purpose  of  This  Census 

Any  military  or  government  civilian  engaged  in  acquisition  cost  estimating,  cost  analysis  and 
modeling,  and/or  cost/risk  assessment,  either  full-time  or  part-time;  or  any  contractor  person¬ 
nel  or  consultant  working  in  similar  positions  for  an  AF  organization  in  support  of  acquisition 
programs;  or  any  authorized  but  vacant  position  supporting  similar  cost  analysis  and  estimat¬ 
ing  (e.g.,  AFSC  65W  positions  in  acquisition  offices).  Individuals  providing  data  in  support  of 
cost  analysis  should  only  be  counted  in  this  category  if  they  participate  in  the  analysis  of  that 
data  to  produce  cost  estimates.  Earned  value  management  activity  should  only  be  counted  in 
this  category  if  it  directly  supports  cost  estimating  activities. 

Please  include  cost  analysts  assigned  to  the  office  who  are  temporarily  elsewhere  for  train¬ 
ing  or  other  TDY. 
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Definitions  of  Requested  Information  on  the  Worksheet  Form 

AF  Organization  &  Office  Symbol:  The  name  of  the  wing,  group,  or  squadron  and  office 
symbol  of  the  office  the  cost  analyst  is  currently  working  with  (or  assigned  to)  (e.g.,  Hq  ESC, 
FMP;  677th  Aeronautical  Systems  Group,  FM). 

ACAT  Program(s):  Enter  the  acronym  for  the  ACAT  program(s)  the  individual  is  work¬ 
ing  on  or  has  worked  on  during  the  past  six  months.  Use  the  acronyms  from  the  list  provided 
in  worksheet  4,  if  applicable,  or  provide  a  full  program  name  if  the  relevant  program  is  not 
included  in  the  list.  You  may  enter  “and  other”  if  the  individual  is  associated  in  a  relatively 
minor  way  with  programs  other  than  those  explicitly  listed.  It  is  especially  important  to  iden¬ 
tify  ACAT  programs  for  which  the  individual  or  position  is  designated  as  lead  cost  estimator. 

Position  Number  (MPCN):  (for  organic  employees  and  vacant  positions  only):  The  posi¬ 
tion  number  (MPCN)  of  the  position  held  by  the  cost  analyst.  If  the  employee’s  position 
has  no  authorized  position  number  (or  the  position  number  is  unknown),  enter  “overhire,” 
“unknown,”  or  other  text  indicating  the  type  of  position.  Please  do  not  leave  this  cell  blank. 
The  MPCN  will  be  used  along  with  the  employee  name  to  link  data  from  personnel  system 
databases  where  possible. 

Status  (for  organic  employees  only):  Enter  “full-time  on  duty,”  “full-time  TDY,  else¬ 
where,”  “Part-time,”  “TDY  for  training,”  or  other  appropriate  text  if  the  employee  is  not  full¬ 
time  and  currently  on  duty. 

Contractor  Organization  (for  contractor  positions  only):  Enter  the  name  of  the  contract¬ 
ing  organization  that  employs  the  contractor/consultant  cost  analyst  (e.g.,  Tecolote,  MITRE). 

Senior,  Mid-Level,  or  Junior  (for  contractor  positions  only):  Enter  one  of  the  three  cat¬ 
egories  to  indicate  the  approximate  level  of  seniority,  expertise,  or  experience  of  the  contractor/ 
consultant  analyst. 

Lead  Estimator  and  for  What  Program(s):  Enter  the  acronym  or  name  of  ACAT  program(s) 
for  which  this  analyst  is  the  lead  cost  estimator  from  the  list  of  ACAT  programs  included  in 
worksheet  4.  It  is  important  that  all  programs  for  which  the  analyst  is  the  lead  estimator  be 
included. 

Mil  Grade  or  Civilian  Pay-Plan-Series-Grade  (for  organic  employees  and  vacant  posi¬ 
tions  only):  Military  grade  for  a  military  cost  analyst  or  position;  pay-plan,  occupational  series 
and  grade  for  a  civilian  cost  analyst  or  position. 

Duty  AFSC  (for  organic  employees  and  vacant  positions  only):  Duty  AFSC  of  the  cost 
analyst  or  authorized  AFSC  for  a  vacant  position. 

Duty  Title  (for  organic  employees  and  vacant  positions  only):  Official  duty  title  for  the 
individual,  or  for  the  position.  (May  not  be  available  for  all  positions). 

Percent  Time  Doing  Acquisition  Cost  Estimating  During  the  Past  Six  Months:  The  per¬ 
centage  of  time  the  cost  analyst  has  worked  on  cost  estimating  and  cost  analysis  tasks  over  the 
last  six  months  (or  since  being  assigned  to  the  current  position,  if  less  than  six  months).  Enter 
the  expected  percentage  of  time  that  would  be  devoted  to  acquisition  cost  estimating  for  vacant 
positions.  Enter  one  of  the  following  five  categories:  100  percent;  75  percent  or  more;  50  per¬ 
cent  or  more;  25  percent  or  more;  or  less  than  25  percent. 

Percent  Time  Doing  EVM  Other  Than  in  Support  of  Cost  Analysis  During  the  Past  Six 
Months:  The  percentage  of  time  the  cost  analyst  has  worked  on  Earned  Value  Management 
work  in  the  past  six  months  other  than  that  counted  as  “acquisition  cost  analysis.”  (N/A  for 
vacant  positions).  Enter  one  of  the  four  following  categories:  75  percent  or  more;  50  percent  or 
more;  25  percent  or  more;  or  less  than  25  percent. 
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Percent  Time  Doing  Other  FM  Work  During  Past  Six  Months:  The  percentage  of  time 
the  cost  analyst  has  worked  on  budgeting,  programming,  or  other  FM  work  in  the  past  six 
months.  Enter  one  of  the  four  following  categories:  75  percent  or  more;  50  percent  or  more;  25 
percent  or  more;  or  less  than  25  percent. 

Total  Years  of  Cost  Analysis  Experience:  The  total  number  of  years  the  individual  has 
worked  in  cost  analysis  positions  for  at  least  50  percent  of  the  time,  including  non-acquisition 
cost  analysis. 

Total  Years  of  Other  Financial  Management  Experience:  The  total  number  of  years  the 
individual  has  worked  in  financial  management  or  earned  value  management  other  than  as  a 
cost  analyst. 

Total  Years  of  Acquisition  Experience:  The  total  number  of  years  the  individual  has 
worked  in  any  area  of  acquisition,  including  positions  both  in  and  outside  of  financial  manage¬ 
ment  and  cost  analysis. 

Highest  Degree  &  Field  of  Study:  Highest  educational  degree  obtained  for  assigned  indi¬ 
viduals  and  contractor  personnel. 

Field  of  Study  for  Highest  Degree:  Educational  field  of  study  for  degree  listed  above,  for 
assigned  individuals  and  contractor  personnel. 

Formal  Cost  Analysis  Training:  Enter  formal  cost  analysis  courses  or  educational  pro¬ 
gram  and  source  for  assigned  individuals  and  contractor  personnel  (e.g.,  APDP  Eevel  III; 
CostProf). 

Certification(s)  and  Source:  Acquisition  or  cost  analysis  certification(s)  obtained  and 
source  of  certification(s),  for  assigned  individuals  and  contractor  personnel. 

Notes:  If  needed,  enter  other  explanatory  information  in  this  cell. 
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Wing,  Group,  and  Squadron  Interviews 


Aeronautical  Systems  Center 

•  ASC/FMC  (Cost  staff) 

•  0677  ATL  GP  (B-2/Training  Systems) 

•  0303  ATL  GP  (Global  Hawk) 

•  0658  ATL  GP  (MQ-1;  MQ-9) 

•  0912OTLGP  (F-15) 

•  0669  ATL  SQ  (CSAR-X) 

•  0478  ATL  GP  (F-22A) 

•  0516  ATL  GP  (C-17A) 


Electronic  Systems  Center 

•  ESC/FMC  (Cost  staff) 

•  350  ELSG  (C2  Devices) 

•  551  EESG  (Early  Warning) 

•  653  EESW  (Global  Information  Grid;  Planning  Integration;  Air 


Traffic) 


653  EESG  (Global  Information  Grid) 

951  EESG  (Mission  Planning) 


Space  and  Missile  Systems  Center 


GPSW 

ERSW 

MSCW 

SCNG 

ISSW 

SYSW 


SMC/EMC 


(Cost  staff) 

Global  Positioning  Systems  Wing 
Eaunch  and  Range  Systems  Wing 
Military  Satellite  Communications  Wing 
Satellite  Control  and  Network  Systems  Group 
Space-Based  Infrared  Systems  Wing 
Space  Superiority  Systems  Wings 
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Protocol  for  Interviews  with  Group  Commanders  and  Wing 
Commanders 


Thank  you  for  your  willingness  to  participate  in  this  study  of  characteristics  and  needs  for  cost 
analysts.  I  am  XXX  and  this  is  YYY  with  the  RAND  Corporation.  We  are  conducting  this 
study  in  collaboration  with  our  sponsor  SAF/AQX  and  AFCAA.  With  us  are  ZZZ. 

Your  participation  in  this  interview  is  voluntary  and  you  may  refuse  to  participate  in 
this  interview  or  answer  any  of  the  questions  asked.  All  information  you  give  us  today  will  be 
kept  strictly  confidential.  We  will  not  provide  or  publish  any  information  that  identifies  you  to 
anyone  outside  of  the  study  team,  except  as  may  be  required  by  law. 

1.  How  many  of  your  group’s  (wing’s)  staff  are  assigned  to  cost  analysis  tasks? 

2.  How  were  these  cost  analysts  assigned  (or  selected)  to  work  on  your  group’s  (wing’s) 
ACAT  programs? 

a.  By  what  criteria  or  considerations? 

3.  How  frequently  do  cost  analysts  rotate  in  and  out  of  your  group  (wing)?  How  frequently 
across  your  group’s  (wing’s)  programs?  Why? 

Adequacy  of  Workforce 

4.  Are  there  enough  cost  analysts  to  effectively  support  your  group’s  (wing’s)  portfolio  of 
ACAT  acquisition  programs? 

a.  If  yes,  any  excess  cost  analysts?  How  many  could  you  do  without? 

b.  If  no,  how  many  additional  cost  analysts  are  needed? 

1.  For  what  (type  of)  programs? 

2.  Military/civilian/FFRDC/other? 

3.  Years  of  experience  in  cost  analysis? 

c.  Why  do  you  say  you  need  these  additional  staff?  What  analytical  cost  func¬ 
tions  do  not  get  done  or  do  not  get  done  as  frequently  as  they  ought  to  be 
done? 

d.  Are  there  specific  cost  analysis  activities  that  you  would  allocate  the  addi¬ 
tional  staff  to  do? 

e.  What  are  the  implications  of  not  having  the  right  number  of  cost  analysts? 
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5.  How  many  cost  estimators,  over  how  many  months,  does  it  take  to  perform  a  thorough 
estimate  of  the  cost  of  a  $xxx-million  new  program? 

6.  How  is  the  quality/accuracy  of  a  cost  estimate  determined? 


Technical  Competencies 

7.  Are  you  satisfied  with  the  technical  competencies  of  your  cost  analysis  staff? 

8.  Currently,  what  kind  of  technical  skill,  experience,  knowledge  or  other  competencies 
would  you  say  are  generally  lacking  or  not  available  among  the  cost  analysts  in  your 
group? 

a.  What  would  get  done  that  is  not  now  done  if  your  cost  analysts 
had  these  competencies? 

h.  What  would  get  done  better  if  your  cost  analysts  had  these 
competencies? 

9.  Do  you  have  the  right  balance  of  skills,  knowledge,  and  experience  among  the  set  of 
cost  analysts  you  currently  have  in  the  group?  The  right  level  of  breadth  and  depth? 


Future  Requirements 

10.  Looking  five  years  ahead,  what  kind  of  different  technical  skills,  experience,  knowledge, 
or  competencies  may  be  needed? 

a.  Why?  What  changes  in  technology,  mix  of  programs  or  other  changes  do 
you  foresee  may  make  this  necessary? 

1 1 .  How  many  cost  analysts  were  hired  specifically  to  work  on  your  group’s  programs  over 
the  past  12  months?  Last  24  months? 

a.  How  difficult  is  it  to  recruit  cost  analysts  in  your  area? 

12.  How  many  cost  analysts  left  your  groups  in  the  past  12  months?  Past  24  months? 

a.  Where  did  they  go? 

13.  What  measures  would  you  suggest  taking  to  improve  the  training  and  performance  of 
cost  analysts? 
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Table  D.1 

Number  of  Cost  Estimators,  by  Type  of  Personnel  and  Center,  2008 


Center 

Military 

Civilian 

Contractor 

Total 

AAC 

6 

13 

7 

26 

ASC 

7 

70 

40 

117 

ESC 

9 

16 

79 

104 

SMC 

17 

25 

49 

91 

OC-ALC 

0 

11 

1 

12 

00-ALC 

0 

8 

6 

14 

WR-ALC 

1 

1 

8 

10 

Total 

40 

144 

190 

374 

SOURCE:  RAND  census,  2008. 


Table  D.2 

Number  of  Authorized  Vacancies  for  Cost  Estimators,  by 
Type  of  Personnel  and  Center,  2008 


Center 

Military 

Civilian 

Total 

AAC 

— 

3 

3 

ASC 

2 

19 

21 

ESC 

5 

5 

10 

SMC 

10 

16 

26 

OC-ALC 

— 

5 

5 

WR-ALC 

— 

9 

9 

Total 

17 

57 

74 

SOURCE:  RAND  census,  2008. 
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Table  D.3 

Cost  Analysts'  Academic  Attainment,  by  Center,  and  Personnel  Type,  2008  (%) 


Center 

Personnel  Type 

Associate 

Bachelor's 

Master's 

MBA 

PhD 

Total 

AAC 

Organic  (n  =  19) 

0 

16 

84 

0 

0 

100 

Contractor  (n  =  6) 

0 

83 

17 

0 

0 

100 

All(n  =  25) 

0 

32 

68 

0 

0 

100 

ASC 

Organic  (n  =  75) 

1 

28 

27 

44 

0 

100 

Contractor  (n  =  39) 

0 

27 

47 

27 

0 

100 

All  (n  =  114) 

1 

28 

33 

38 

0 

100 

ESC 

Organic  (n  =  25) 

0 

44 

40 

6 

0 

100 

Contractor  (n  =  79) 

0 

43 

25 

30 

1 

100 

All  (n  =  104) 

0 

43 

29 

27 

1 

100 

SMC 

Organic  (n  =  41) 

0 

31 

69 

0 

0 

100 

Contractor  (n  =  49) 

0 

49 

49 

0 

2 

100 

All  (n  =  90) 

0 

42 

57 

0 

1 

100 

OC-ALC 

Organic  (n  =  10) 

0 

30 

60 

10 

0 

100 

Contractor  (n  =  1) 

0 

0 

0 

100 

0 

100 

All  (n  =  11) 

0 

27 

55 

18 

0 

100 

OO-ALC 

Organic  (n  =  8) 

0 

75 

0 

25 

0 

100 

Contractor  (n  =  6) 

0 

17 

83 

0 

0 

100 

All  (n  =  14) 

0 

50 

36 

14 

0 

100 

WR-ALC 

Organic  (n  =  2) 

0 

0 

50 

50 

0 

100 

Contractor  (n  =  6) 

0 

0 

33 

50 

18 

100 

All  (n  =  8) 

0 

0 

38 

50 

13 

100 

All 

Organic  (n  =  180) 

1 

32 

44 

23 

0 

100 

Contractor  (n  =  186) 

0 

39 

40 

20 

1 

100 

All  (n  =  366) 

0 

36 

42 

21 

1 

100 

SOURCE:  RAND  census,  2008. 

NOTE:  Percentages  may  not  add  to  100  because  of  rounding. 
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Table  D.4 

Cost  Analysts'  Years  of  Experience  Doing  Cost  Estimation,  by  Center  and  Personnel  Type,  2008  (%) 


Center 

Personnel  Type 

0-2 

3-5 

6-10 

11-20 

21+ 

Total 

AAC 

Organic  (n  =  19) 

16 

26 

16 

26 

16 

100 

Contractor  (n  =  6) 

33 

0 

0 

17 

50 

100 

All(n  =  25) 

20 

20 

12 

24 

24 

100 

ASC 

Organic  (n  =  75) 

39 

9 

23 

19 

11 

100 

Contractor  (n  =  34) 

3 

15 

21 

21 

41 

100 

All  (n  =  109) 

28 

11 

22 

19 

20 

100 

ESC 

Organic  (n  =  25) 

28 

24 

28 

4 

16 

100 

Contractor  (n  =  76) 

13 

17 

26 

25 

18 

100 

All  (n  =  101) 

17 

19 

27 

20 

18 

100 

SMC 

Organic  (n  =  40) 

43 

23 

10 

18 

8 

100 

Contractor  (n  =  48) 

19 

13 

23 

33 

13 

100 

All  (n  =  88) 

30 

17 

17 

26 

10 

100 

OC-ALC 

Organic  (n  =  11) 

18 

27 

9 

46 

0 

100 

Contractor  (n  =  1) 

0 

0 

0 

0 

100 

100 

All  (n  =  12) 

17 

25 

8 

42 

8 

100 

OO-ALC 

Organic  (n  =  8) 

13 

25 

50 

0 

13 

100 

Contractor  (n  =  6) 

17 

17 

17 

33 

17 

100 

All  (n  =  14) 

14 

21 

36 

14 

14 

100 

WR-ALC 

Organic  (n  =  2) 

50 

0 

0 

0 

50 

100 

Contractor  (n  =  7) 

0 

0 

29 

29 

43 

100 

All  (n  =  9) 

11 

0 

22 

22 

44 

100 

All 

Organic  (n  =  180) 

33 

18 

19 

18 

11 

100 

Contractor  (n  =  178) 

13 

14 

23 

26 

24 

100 

All  (n  =  358) 

23 

16 

21 

22 

17 

100 

SOURCE:  RAND  census,  2008. 

NOTE;  Percentages  may  not  add  to  100  because  of  rounding. 
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Table  D.5 

Time  Spent  Doing  Cost  Estimation  Over  Previous  Six  Months,  by  Center  and  Personnel  Type, 
2008  (%) 


Center  Personnel  Type 

Zero 

Less  Than 
25 

25-49 

50-74 

75-99 

100 

Total 

AAC  Organic  (n  =  19) 

16 

5 

32 

5 

11 

32 

100 

Contractor  (n  =  7) 

0 

14 

0 

0 

14 

71 

100 

All  (n  =  26) 

12 

8 

23 

4 

12 

42 

100 

ASC  Organic  (n  =  77) 

0 

17 

20 

7 

12 

46 

100 

Contractor  (n  =  40) 

0 

13 

8 

13 

18 

50 

100 

All  (n  =  117) 

0 

15 

15 

9 

14 

47 

100 

ESC  Organic  (n  =  25) 

0 

4 

8 

12 

20 

56 

100 

Contractor  (n  =  79) 

0 

4 

0 

5 

70 

22 

100 

All  (n  =  104) 

0 

4 

2 

7 

58 

30 

100 

SMC  Organic  (n  =  42) 

12 

17 

14 

21 

12 

24 

100 

Contractor  (n  =  49) 

2 

12 

0 

10 

53 

22 

100 

All  (n  =  91) 

7 

14 

7 

15 

34 

23 

100 

OC-ALC  Organic  (n  =  11) 

9 

18 

18 

18 

36 

0 

100 

Contractor  (n  =  1) 

0 

0 

0 

0 

100 

0 

100 

All  (n  =  12) 

8 

17 

17 

17 

42 

0 

100 

OO-ALC  Organic  (n  =  8) 

0 

88 

0 

13 

0 

0 

100 

Contractor  (n  =  6) 

0 

0 

0 

0 

0 

100 

100 

All  (n  =  14) 

0 

50 

0 

7 

0 

43 

100 

WR-ALC  Organic  (n  =  2) 

0 

50 

0 

0 

50 

0 

100 

Contractor  (n  =  8) 

0 

75 

13 

13 

0 

0 

100 

All  (n  =  10) 

0 

70 

10 

10 

10 

0 

100 

All  Organic  (n  =  184) 

5 

17 

17 

12 

14 

35 

100 

Contractor  (n  =  190) 

1 

10 

2 

8 

47 

31 

100 

All  (n  =  374) 

3 

13 

9 

10 

32 

33 

100 

SOURCE:  RAND  census,  2008. 

NOTE:  Percentages  may  not  add  to  100  because  of  rounding. 
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Table  D.6 

Lead  Cost  Estimators,  by  Center  and  Personnel  Type,  2008  (%) 


Center 

Military 

Civilian 

Contractor 

Total 

AAC  (n  =  11) 

18 

64 

18 

100 

ASC  (n  =  54) 

4 

57 

39 

100 

ESC  (n  =  68) 

9 

19 

72 

100 

SMC  (n  =  25) 

56 

44 

0 

100 

OC-ALC  (n  =  1) 

0 

0 

100 

100 

OO-ALC  (n  =  7) 

0 

57 

43 

100 

WR-ALC  (n  =  3) 

0 

0 

100 

100 

All  (n  =  169) 

14 

39 

47 

100 

SOURCE:  RAND  census,  2008. 

NOTE:  Percentages  may  not  add  to  100  because  of  rounding. 

Table  D.7 

Lead  Estimators'  Years  of  Experience  in  Cost  Estimation,  by  Center,  2008  (%) 

Center  Personnel  Type 

0-2 

3-5 

6-10 

11-20 

21-h 

Total 

AAC  Organic  (n  =  9) 

11 

44 

— 

11 

33 

100 

Contractor  (n  =  2) 

— 

— 

— 

50 

50 

100 

All  (n  =  11) 

9 

36 

— 

18 

36 

100 

ASC  Organic  (n  =  32) 

34 

9 

16 

25 

16 

100 

Contractor  (n  =  21) 

— 

10 

19 

33 

38 

100 

All  (n  =  53) 

21 

9 

17 

28 

25 

100 

ESC  Organic  (n  =  19) 

26 

26 

26 

11 

11 

100 

Contractor  (n  =  49) 

4 

14 

25 

35 

22 

100 

All  (n  =  68) 

10 

17 

25 

28 

19 

100 

SMC  Organic  (n  =  25) 

52 

24 

4 

20 

— 

100 

Contractor  (n  =  0) 

— 

— 

— 

— 

— 

— 

All  (n  =  25) 

52 

24 

4 

20 

— 

100 

OC-ALC  Organic  (n  =  0) 

— 

— 

— 

— 

— 

— 

Contractor  (n  =  1) 

— 

— 

— 

— 

100 

100 

All  (n  =  1) 

— 

— 

— 

— 

100 

100 

OO-ALC  Organic  (n  =  4) 

— 

50 

50 

— 

— 

100 

Contractor  (n  =  3) 

33 

— 

— 

67 

— 

100 

All  (n  =  7) 

14 

29 

29 

29 

— 

100 

WR-ALC  Organic  (n  =  0) 

— 

— 

— 

— 

— 

— 

Contractor  (n  =  2) 

— 

— 

50 

— 

50 

100 

All  (n  =  2) 

— 

— 

50 

— 

50 

100 

All  Organic  (n  =  89) 

34 

22 

18 

15 

10 

100 

Contractor  (n  =  78) 

4 

12 

22 

39 

28 

100 

All  (n  =  167) 

20 

17 

20 

24 

19 

100 

SOURCE:  RAND  census,  2008. 

NOTE:  Percentages  may  not  add  to  100  because  of  rounding. 
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Table  D.8 

Lead  Estimators'  Time  Spent  Doing  Cost  Estimation,  by  Center  and  Personnel  Type,  2008  (%) 


Center 

Personnel  Type 

Zero 

Less  Than 
25 

25-49 

50-74 

75-99 

100 

Total 

AAC 

Organic  (n  =  9) 

— 

— 

11 

11 

11 

67 

100 

Contractor  (n  =  2) 

— 

— 

— 

— 

50 

50 

100 

All  (n  =  11) 

— 

— 

9 

9 

18 

64 

100 

ASC 

Organic  (n  =  33) 

— 

12 

9 

12 

21 

47 

100 

Contractor  (n  =  21) 

— 

5 

5 

19 

19 

52 

100 

All  (n  =  54) 

— 

9 

7 

15 

20 

48 

100 

ESC 

Organic  (n  =  19) 

— 

— 

5 

16 

21 

58 

100 

Contractor  (n  =  49) 

— 

2 

— 

6 

71 

20 

100 

All  (n  =  68) 

— 

2 

2 

9 

57 

31 

100 

SMC 

Organic  (n  =  25) 

4 

8 

12 

32 

20 

24 

100 

Contractor  (n  =  0) 

— 

— 

— 

— 

— 

— 

— 

All  (n  =  25) 

4 

8 

12 

32 

20 

24 

100 

OC-ALC 

Contractor  (n  =  1) 

— 

— 

— 

— 

100 

— 

100 

OO-ALC 

Organic  (n  =  4) 

— 

75 

— 

25 

— 

— 

100 

Contractor  (n  =  3) 

— 

— 

— 

— 

— 

100 

100 

All  (n  =  7) 

— 

43 

— 

14 

— 

43 

100 

WR-ALC 

Contractor  (n  =  3) 

— 

33 

33 

33 

— 

— 

100 

All 

Organic  (n  =  90) 

1 

10 

9 

19 

19 

42 

100 

Contractor  (n  =  79) 

— 

4 

3 

10 

52 

32 

100 

All  (n  =  169) 

1 

7 

6 

15 

34 

37 

100 

SOURCE:  RAND  Census,  2008. 

NOTE:  Percentages  may  not  add  to  100  because  of  rounding. 
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